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A NOTE  TO  THE  READER 


On  August  23,  1988,  the  Omnibus  Trade  and  Competitiveness  Act  (Public  Law 
100-418)  was  signed  into  law.  The  Act  created  the  National  Institute  of 
Standards  and  Technology  (NIST)  from  the  National  Bureau  of  Standards  (NBS). 
Since  1901,  NBS  has  served  as  the  Nation’s  central  laboratory  for  developing 
and  disseminating  measurement  standards  and  scientific  data. 

Central  to  the  competitiveness  portion  of  the  Act  was  the  expanded  mission  of 
NIST.  The  Institute’s  new  responsibilities  augment  and  build  on  the  technical 
expertise  of  NBS,  the  only  Federal  laboratory  with  a specific  mission  to 
support  U.S.  industry. 

NIST  will  maintain  the  traditional  functions  of  NBS  and  will  continue  to  offer 
the  full  array  of  measurement  and  quality  assurance  services  that  were 
provided  by  NBS.  These  services  include  calibration  services,  standard 
reference  materials,  standard  reference  data,  and  measurement  assurance 
p rograms . 
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PREFACE 


The  Center  for  Computing  and  Applied  Mathematics,  National  Engineering 
Laboratory,  National  Institute  of  Standards  and  Technology  (NIST)  is 
conducting  a research  effort  under  the  sponsorship  of  the  Federal  Emergency 
Management  Agency  (FEMA)  to  develop  a set  of  guidelines  which  promotes  both  a 
critical  review  and  unbiased  evaluation  of  FEMA  models  and  data  bases. 

Large-scale  models  and  data  bases  are  key  informational  resources  for 
FEMA.  The  type  and  nature  of  FEMA  models  and  data  bases,  however,  are  perhaps 
unique  within  the  Federal  government  in  that  they  are  often  concerned  with 
severe  changes  or  disruptions  ranging  from  limited  effects  to  extremes.  In 
order  to  carry  out  its  emergency  missions,  it  is  necessary  for  FEMA  to 
determine  which  models,  modeling  techniques  and  data  bases  are  appropriate  for 
what  purposes  and  which  ones  need  modification,  updating  and  maintenance.  The 
development  of  evaluation  guidelines  is  therefore  of  direct  benefit  to  FEMA  in 
discharging  its  emergency  management  duties. 

FEMA's  Dynamic  General  Equilibrium  Model  (DGEM)  was  subjected  to  a 
critical  review  and  evaluation  of  its  capabilities  as  a military  mobilization 
model  to  test  and  illustrate  the  approach  recommended  for  evaluating  emergency 
management  models  and  data  bases.  DGEM  is  ideally  suited  for  this  purpose 
because  it  was  designed  to  deal  with  economic  and  policy  impacts  of  energy  and 
resource  supply  (disruptions),  catastrophic  losses  of  resources  and  recovery 
from  enemy  attack,  and  demand  and  production  surges  and  resource  constraints 
during  a military  mobilization.  It  is  hoped  that  these  evaluation  guidelines 
and  their  application  to  DGEM  will  prove  useful  to  emergency  managers  and 
decision  makers  both  in  the  application  of  these  guidelines  to  other  emergency 
management  models  and  data  bases  and  in  their  understanding  and  appreciation 
of  the  capabilities  provided  in  DGEM. 
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EXECUTIVE  SUMMARY 


The  purpose  of  this  report  is  twofold.  First,  it  is  designed  to  provide 
the  reader  with  a generic  set  of  guidelines  which  can  be  used  to  evaluate 
large-scale,  computer-based  models  and  data  bases.  Second,  the  guidelines  are 
illustrated  through  a critical  evaluation  of  the  Dynamic  General  Equilibrium 
Model  (DGEM).  DGEM  is  currently  being  used  by  the  Federal  Emergency 
Management  Agency  (FEMA)  to  analyze  a variety  of  emergency  management 
problems.  The  evaluation  of  DGEM  serves  both  as  a step-by-step  procedure  for 
conducting  an  in-depth  model  evaluation  and  as  an  introduction  to  a 
non-proprietary  model  which  has  broad  applicability  to  the  analysis  of 
macroecomic  issues. 


Many  of  the  models  currently  in  use  within  government  and  the  private 
sector  are  by  definition  complex  and  large-scale.  Consequently,  the  ability 
to  produce  accurate  and  realistic  estimates  of  alternative  courses  of  action 
has  become  a most  challenging  task.  Furthermore,  many  models  are  dependent  on 
data  bases  that  are  generated  and  maintained  by  diverse  organizations,  with 
each  data  base  having  a raison  d’etre  that  may  or  may  not  be  relevant  to  the 
problem  under  review.  It  should  then  be  clear  that  the  acceptance  of  model 
outputs  as  inputs  to  the  decision-making  process  without  investigating  the 
validity  of  their  generation  is  a luxary  that  the  community  of  model  users  can 
no  longer  afford. 

Over  the  past  15  years  a body  of  research  has  been  developed  that 
addresses  the  concern  of  model  users  who,  upon  being  presented  with  the 
results  of  a computer-based  model  analysis,  must  decide  whether  the  results 
are  accurate  enough  or  appropriate  for  the  problem  under  review.  This 
research  - termed  model  evaluation  - has  contributed  greatly  to  our  ability  to 
understand  better  the  role  of  modeling  in  policy-oriented  decision  making. 

Model  evaluation  may  take  on  a variety  of  forms  and  levels  of  complexity. 
However,  any  evaluation  procedure  has  at  least  the  following  essential  phases: 

1.  Obtain  a clear  and  comprehensive  statement  of  user  requirements  and 
objectives  pertaining  to  the  application  of  the  model; 

2.  Generate  appropriate  information  about  the  model  design  and  perfor- 
mance pertaining  to  user  requirements;  and 


3. 


Evaluate  model  attributes  and  properties  according  to  predetermined 
criteria  of  performance  required  by  the  user. 


These  phases  are  characterized  by  the  following  activities: 

Determine  user  requirements  and  objectives. 

0 Based  on  user  requirements,  develop  questions  to  be  answered  about 
model  performance  and  identify  problems  to  be  resolved  or  analyses  to 
be  performed. 
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0 Ascertain  from  the  questions  any  problems  for  which  specialized  tech- 
niques are  to  be  used  to  generate  information  needed  about  model 
design  or  performance  relevant  to  the  intended  application  of  the 
model  by  the  user. 

° Perform  the  necessary  tasks  to  generate  the  needed  information  for 
the  evaluation. 

0 Establish  criteria  for  the  model  evaluation. 

0 Conduct  a formal  evaluation  to  judge  the  integrity  of  the  model 
relative  to  its  intended  use. 

The  first  five  chapters  of  this  report  focus  on  the  development  of  a 
generic  evaluation  procedure.  Additional  material  on  the  general  topic  of 
model  evaluation  is  presented  in  Appendices  A,  B and  C.  The  remainder  of  the 
report  illustrates  how  the  procedure  was  applied  to  DGEM . 


DGEM  was  selected  for  evaluation  primarily  for  its  unique  approach  to 
modeling  emergency  situations.  The  availability  of  documentation  was  also  a 
major  factor.  We  wish  to  alert  the  reader  to  the  importance  of  documentation 
in  any  model  evaluation  effort  because  it  is  a means  through  which  performance 
against  user  requirements  can  be  measured. 


From  a review  of  the  DGEM  documentation,  it  became  clear  that  much  care 
went  into  the  design,  development  and  testing  of  the  model.  Furthermore,  the 
model's  documentation  is  quite  complete  and  should  be  sufficient  to:  (1)  set 

it  up  on  the  host  system;  (2)  execute  a base-case  simulation;  (3)  interpret 
the  results  of  the  base-case  simulation;  and  (4)  create,  run,  and  interpret 
user-specified  simulations. 

The  analysis  of  the  U.S.  economy  during  a military  mobilization  is  used 
to  illustrate  the  generic  evaluation  procedure.  This  approach  was  taken  for 
two  reasons.  First,  a military  mobilization  represents  a major  perturbation 
about  DGEM ' s "business  as  usual"  base-case  simulation.  Consequently,  any 
weaknesses  of  the  model  (e.g.,  instabilities,  implausible  results,  etc.) 
should  be  revealed  by  such  a perturbation.  Second,  a military  mobilization 
requires  an  explicit  treatment  of  the  joint  interactions  of  several  factors 
(e.g.,  international  trade,  the  business  cycle,  available  capacity, 
investment,  etc.). 


The  formulation  of  the  mobilization  see 
DGEM  was  a two-stage  process.  In  the  first, 
constructed  which  made  explicit  the  perturba 
values  in  the  DGEM  data  base.  The  mobilizat 
of  reference  for  how  the  joint  interactions 
path  of  the  national  economy.  In  the  second 
combination  subject  to  an  experimental  desig 
detail  cetain  patterns  which  were  uncovered 


nario  used  in  the  evaluation  of 
a mobilization  baseline  was 
tion  about  the  business  as  usua 
ion  baseline  also  provided  a po 
of  the  perturbation  affected  th 
, six  key  variables  were  varied 
n.  This  enabled  us  to  explore 
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For  the  case  at  hand,  DGEM  was  found  to  produce  solutions  which  were  both 
realistic  and  internally  consistent  from  an  economic  perspective.  This  is 
especially  important  in  light  of  the  results  of  the  structured  sensitivity 
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analysis  because  the  model  was  sufficiently  robust  to  produce  mea 
estimtaes  of  key  economic  variables  even  though  many  of  the  "inpu 
were  significantly  different  from  their  business  as  usual  values. 
DGEM T s analytical  approach  and  detailed  reporting  format  enabled 
illustrate  how  techniques  used  in  a model  evaluation  can  be  appli 
studies  with  policy  implications. 
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INTRODUCTION 


1 . 


1.1  BACKGROUND 

The  subject  of  model  and  data  base  evaluation  has  grown  rapidly  in 
importance  during  the  last  15  years.  This  trend  is  due  primarily  to  a shift  in 
the  type  of  models  being  developed  by  members  of  the  operations  research  and 
management  science  communities.  During  the  period  immediately  following  World 
War  II,  numerous  large-scale  models  were  developed  and  used  by  management  as 
aids  to  decision  making.  In  many  cases  these  models  dealt  with 

production-oriented  problems.  Model  evaluation  was  thus  fairly  straightforward, 
in  that  a recommended  solution  could  be  implemented  and  the  response  measured 
against  predictions.  However,  as  one  moves  from  engineering-based, 
production-oriented  problems  to  the  realm  of  policy  analysis,  the  problem 
becomes  more  complex.  No  decision  maker  faced  with  implementing  a major  policy 
is  in  a position  to  control  all  of  the  variables  to  attain  the  desired  response. 
Consequently,  to  compensate  for  this  situation,  the  types  of  models  used  to 
provide  information  to  decision  makers  have  become  both  large  and  complex. 
Although  many  traditional  mathematical  tools  are  used  in  such  models  to  make  the 
analysis  tractable,  the  implications  of  their  use  must  be  carefully  stated  if 
the  model's  results  are  to  be  useful,  usable  and  used  by  the  decision  maker. 

The  problems  associated  with  the  use  of  large-scale  models  in  the  policy 
arena  have  led  a number  of  researchers  to  advocate  the  development  of 
well-defined  sets  of  model  and  data  base  evaluation  guidelines.  One  of  the  best 
known  positions,  advocated  by  Saul  I.  Gass,-*-  stresses  the  role  of  evaluation  as 
an  integral  part  of  the  building  and  use  of  any  large-scale,  policy-oriented 
model.  Gass  provides  three  reasons  for  advocating  this  form  of  evaluation: 

(1)  in  many  cases,  the  ultimate  decision  maker  is  far  removed  from  the 
modeling  process  and  needs  a basis  for  deciding  when  to  accept  the 
model's  results; 

(2)  users  of  a model  developed  for  others  must  be  able  to  obtain  a clear 
statement  of  the  applicability  of  the  model  to  their  problem  area; 
and 

(3)  it  is  difficult  to  assess  the  impact  of  a model's  assumptions,  data 
availability,  and  other  elements  on  the  model's  structure  and 
results  without  a formal,  independent  evaluation.- 


^Gass,  S.I.  "Decision-Aiding  Models,  Validation,  Assessment,  and  Related  Issues 
for  Policy  Analysis,"  Operations  Research,  Vol.  31,  No.  4,  1983,  pp . 603-631  . 

“It  is  important  to  recognize  that  model  evaluation  is  not  model  certification. 
Certification  commonly  refers  to  a guarantee  that  the  model  yields  outputs  and 
results  that  are  suitably  accurate  for  a particular  application.  This  is  an 
unattainable  goal  for  the  policy-oriented  models  that  are  of  concern  here. 
Evaluation,  on  the  other  hand,  acknowledges  this  limitation,  and  seeks  to 
improve  the  model's  usefulness  by  identifying  as  explicitly  as  possible,  its 
strengths,  weaknesses,  and  appropriate  uses. 
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Since  this  report  focuses  on  the  subject  of  model  and  data  base 
evaluation,  it  is  first  useful  to  introduce  and  define  a few  key  terms.  The 
literature  on  model  and  data  base  evaluation  refers  to  the  following  key 
procedures:  (1)  documentation;  (2)  verification;  and  (3)  validation.  Since 

many  authors  are  writing  on  this  subject  area, ^ there  is  often  disagreement  on 
the  exact  definition  of  each  term.  A definition  of  each  term  is  therefore 
given  here  in  order  to  promote  an  understanding  of  the  topics  which  follow. 

Documentation  refers  to  the  written  description  of  the  model.  In 
practice  a variety  of  reports  comprise  the  documentation  for  the  model.  These 
reports  include  such  information  as: 

(1)  precise  statements  of  what  the  model  is  supposed  to  do; 

(2)  the  mathematical/ logical  definitions,  assumptions,  and  formulation 
of  the  problem;  and 

(3)  a complete  set  of  operating  instructions  for  the  user,  including 
current  inputs,  outputs,  test  cases,  options  and  limitations. 

Among  the  major  purposes  of  documentation  are:  (1)  to  promote  the 
intelligent  use  of  the  model;  (2)  to  provide  a technical  knowledge  base;  and 
(3)  to  broaden  the  class  of  users.  Although  documentation  is  not  designed  for 
model  evaluation  per  se,  it  is  the  cornerstone  of  any  evaluation  effort. 
Without  documentation  a model  can,  at  best,  be  considered  a black  box. 

In  most  cases  the  documentation  describes  what  the  model  should  do;  but 
may  not  necessarily  describe  what  it  really  does.  The  truth  of  the  previous 
statement  rests  on  the  recognition  that  the  documentation  describes  both  the 
abstract  model  and  the  operational  model.  Since  the  operational  model  is 
embodied  in  a set  of  computer  code,  referred  to  hereafter  as  the  program,  the 
relationship  described  in  the  documentation  may  be  imperfect.  It  is  therefore 
necessary  to  ensure  that  the  program: 

(1)  describes  accurately  the  model  as  designed; 

(2)  is  properly  mechanized  on  the  computer;  and 

(3)  runs  as  intended. 

The  process  of  documenting  the  relationship  between  the  abstract  and 
operational  model  is  referred  to  as  verification. 


^-For  a review  of  the  literature  on  model  and  data  base  evaluation 
see:  Chapman,  R.E.,  R.G.  Hendrickson  and  S.F.  Weber.  A Survey  of  Techniques 
for  Evaluating  Emergency  Planning  Models  and  Data  Bases.  Gaithersburg,  MD: 
National  Bureau  of  Standards,  NBSIR  84-2963,  1984. 
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The  third  term,  validation , refers  to  the  correspondence  between  the 
outputs  of  the  model  and  the  real-world  situation  it  was  designed  to  simulate. 
Given  the  complex  nature  of  policy-oriented  problems  it  is  useful  to 
differentiate  between  two  forms  of  validation.  The  first,  objective 
validation,  is  primarily  concerned  with  model-subject  fidelity  and  the  degree 
to  which  the  model  represents  reality.  The  second,  subjective  validation,  is 
based  on  the  intended  use  of  the  model  and  the  extent  to  which  the  model 
satisfies  the  needs  of  the  user.  For  cases  in  which  models  are  used  to  assess 
the  consequences  of  alternative  policies,  an  objective  validation  is  usually 
impossible.  This  is  because  constraints  on  available  data  and  computer 
resources  may  require  substantial  simplification  of  the  model  design,  so  that 
the  model  focuses  only  on  certain  aspects  of  the  situation  which  are  of 
crucial  importance  to  the  decision  maker.  When  this  view  is  taken,  the 
relevance  of  subjective  validation  to  the  policy  arena  becomes  evident. 


1.2  PURPOSE 

The  purpose  of  this  document  is  twofold.  First,  it  describes  four  major 
classes  of  analytical  techniques  which  are  capable  of  addressing  the 
quantitative  issues  of  model  and  data  base  evaluation.  These  techniques  are 
designed  to  generate  information  which  can  be  used  to  test  the  degree  to  which 
a particular  model  or  data  base  satisfies  a specific  set  of  user  requirements. 
Second,  this  document  focuses  on  how  user  requirements  are  determined  and  how 
a test  against  these  requirements  is  performed.  This  approach  is  taken  to 
illustrate  how  the  four  classes  of  analytical  techniques  provide  the 
information  required  to  perform  an  in-depth  evaluation. 

1.3  SCOPE  AND  APPROACH 


Model  evaluation  may  take  a variety  of  forms  and  levels  of  complexity. 
Irrespective  of  the  form  and  level,  however,  a thorough  evaluation  procedure 
is  based  on  three  essential  phases: 

(1)  obtaining  a clear  and  comprehensive  statement  of  user  requirements 
and  objectives  pertaining  to  the  application  of  the  model; 

(2)  generating  appropriate  information  about  the  model  design  and 
performance  pertaining  to  user  requirements;  and 

(3)  evaluating  model  attributes  and  properties  according  to  predetermined 
criteria  of  performance  required  by  the  user. 

The  last  phase  includes  an  assessment  of  the  model's  qualifications  as  a tool 
in  the  desired  policy  analysis.  These  three  phases  may  be  formally 
characterized  by  the  following  activities: 
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(1)  determine  user  requirements  and  objectives; 

(2)  based  on  the  statement  of  user  requirements,  develop  questions  to  be 
answered  about  model  performance  and  identify  problems  to  be  resolved 
or  analyses  to  be  performed; 

(3)  ascertain  from  the  questions  any  problems  for  which  analytical 
techniques  are  to  be  used  to  generate  information  needed  about  model 
design  or  performance  relevant  to  the  intended  application  of  the 
model  by  the  user; 

(4)  perform  the  necessary  tasks  to  generate  the  needed  information  for 
the  evaluation; 

(5)  establish  criteria  for  the  model  evaluation;  and 

(6)  conduct  a formal  evaluation  to.  judge  the  integrity  of  the  model 
relative  to  its  intended  use. 

These  activities  emphasize  the  importance  of  a thorough  and  comprehensive 
statement  of  user  requirements  to  the  process  and  to  the  success  or  failure  of 
the  evaluation.  The  way  in  which  these  activities  are  integrated  into  an 
evaluation  procedure  is  summarized  in  Figure  1.1.  It  is  important  to  note 
that  the  three  major  components  (user  requirements,  information  development 
and  generation,  and  evaluation  of  test  results)  shown  in  Figure  1.1,  comprise 
a framework  for  the  NBS  model/data  base  evaluation  guidelines. 

This  report  focuses  on  the  central  role  played  by  a comprehensive  set  of 
analytical  techniques  in  the  development  and  generation  component  of  the 
evaluation  procedure  which  appears  at  the  center  of  Figure  1.1.  Specific 
analytical  techniques  are  described  in  Part  I,  Chapters  2 through  5.  Part  II, 
Chapters  6 through  10,  serves  to  relate  these  techniques  to  the  framework  of 
structural  information  and  analysis  which  supports  an  in-depth  model/data  base 
evaluation . 

Chapter  2,  Descriptive  Analysis,  focuses  on  the  development  and 
organization  of  background  information  about  the  model,  which  includes  the 
motivation  for  the  model  creation,  its  design  and  theory,  and  the  conditions 
under  which  it  was  developed.  The  chapter  also  develops  a framework  through 
which  the  real  world  problem  may  be  related  to  the  conceptual  model  used  to 
formulate  the  mathematical  model  and  the  resulting  operational  model. 

Chapter  3,  Program  Verification  and  Analysis,  treats  the  importance  of 
the  clarity,  completeness,  and  adequacy  of  the  model’s  documentation.  The 
details  are  realized  from  three  activities:  (1)  an  in-depth  review  of 

documentation  format,  structure,  and  content;  (2)  a review  of  the  source  code; 
and  (3)  an  assessment  of  experiences  in  using  the  model. 

Chapter  4,  Data  Audit,  discusses  the  evaluation  requirements  as  they 
pertain  to:  (1)  review  and  clarification  of  data  definitions;  (2)  analysis  of 

the  nature  and  structure  of  the  data  base;  (3)  analysis  of  the  file 
computerization  and  accessibility;  (4)  determination  of  data  sources;  and 
(5)  analysis  of  the  quality  of  selected,  important  data  by  estimating 
uncertainties  formulated  as  bounds  on  the  data  variability. 
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Chapter  5,  Sensitivity  Analysis,  describes  the  role  of  sensitivity 
analysis  in  model  evaluation  and  its  contribution  to  quantitative  and 
qualitative  information  development  relevant  to  the  needs  of  the  model 
assessment  process. 

Chapter  6 establishes  a set  of  four  user  requirements  which  provide  the 
means  for  evaluating  DGEM . Requirement  1 focuses  on  model  usage  (i.e.,  the 
operational  model).  Requirement  2 addresses  the  clarity  and  completeness  of 
the  documentation  for  the  conceptual  and  mathematical  models.  Requirement  3 
focuses  on  the  relationships  between  the  National  Income  and  Product  Accounts 
(NIPA)  and  the  model’s  output.  Requirement  4 is  concerned  with  a variety  of 
economic-technical  attributes. 

Chapter  7 addresses  the  issues  raised  in  Chapters  2 and  3.  Background 
informatin,  which  serves  to  define  the  motivation  for  the  model  and  its 
theoretical  under  pinnings,  is  presented  first.  The  focus  is  then  shifted  to 
the  DGEM  software,  including:  (1)  a review  of  the  DGEM  source  code;  (2)  the 
implementation  of  the  model  on  the  NBS  computer  system;  (3)  enhancements  which 
were  made  to  the  basic  code;  and  (4)  a report  on  the  evaluator's  hands-on 
experiences . 

Chapter  8 focuses  on  an  audit  of  the  DGEM  data  base.  Material  is 
presented  which  documents  the  extensive  testing  that  was  performed  on  the  DGEM 
data  base  by  its  developers  (e.g.,  comparisons  with  realized  values, 
assumptions  involved  in  projections  up  through  the  year  2000,  etc.)  as  well  as 
independent  studies  (e.g.,  concerns  about  substitution  elasticity  estimates, 
the  treatment  of  capital  aggregation,  etc.). 

Chapter  9 provides  information  on  the  mobilization  scenario  used  in  the 
evaluation  of  DGEM.  The  mobilization  scenario  analyzed  with  DGEM  is  patterned 
after  an  unclassified  scenario  approved  by  DoD.  Basically,  the  scenario 
indicates  a conventional  war  of  three  years  duration  in  Europe,  the  Persian 
Gulf  and  Korea.  The  period  of  the  mobilization  follows  that  of  the 
unclassified  DoD  scenario  and  is  assumed  to  be  1983  through  1986.  The  first 
year  represents  a period  of  buildup  prior  to  the  onset  of  hostilities.  For 
each  year  of  the  mobilization,  values  for  key  economic  variables  were 
specified.  Once  the  "mobilization  baseline"  had  been  established,  a 
structured  sensitivity  analysis  was  performed. 

Chapter  10  provides  a summary  of  major  findings.  Material  is  presented 
which  compares  critically  information  generated  in  the  model  evaluation 
process  against  user  requirements.  This  comparison  enables  us  to  conclude 
that  DGEM  is  both  an  appropriate  and  useful  model  for  analyzing  the  types  of 
problems  anticipated  during  a military  mobilization. 

1 . 4 A PRIM ER  ON  EMERGENCY  MANAGEMENT:  PROBLEMS,  POLICIES  AND  MODE L S 


The  ability  of  public  officials  to  respond  correctly  to  an  emergency 
situation  is  conditioned  by  their  being  able  to  recognize  that  a particular 
type  of  emergency  has  occurred,  and  to  initiate  predetermined  and  ad  hoc 
actions  designed  to  alleviate  the  situation.  Some  localities  have  established 
contingency  plans  in  the  area  of  public  safety.  Many  metropolitan  areas,  and 
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local  councils  of  governments,  have  centralized  the  process  by  which  an 
emergency  is  declared  and  resources  positioned  for  action  and  response  (e.g., 
during  a hurricane  or  blizzard).  Some  localities  have  functioning  command 
centers  that  enable  them  to  pool,  coordinate  and  allocate  emergency  resources. 
In  general,  however,  the  management  of  the  response  to  an  emergency  tends  to 
be  reactive  instead  of  proactive.  Other  aspects  of  emergency  response  are: 

(1)  the  need  to  determine  appropriate  guidelines  for  response  actions  and 
allocations  of  resources  under  changing  conditions;  (2)  the  training  of 
emergency  managers;  and  (3)  the  development  of  computer-based  decision  aids 
for  use  by  emergency  managers. 

From  a national  perspective,  the  Federal  Emergency  Management  Agency 
(FEMA)  has  an  important  and  direct  role  in  ensuring  that  our  citizens  and 
their  property  are  given  the  best  protection  under  all  emergencies.  FEMA’s 
major  activity  areas  center  around  four  policy  issues.  The  first,  mitigation, 
consists  of  those  activities  which  eliminate  or  reduce  the  probability  of 
occurrence  of  an  emergency.  It  also  includes  long-term  activities  which 
reduce  the  effects  of  unavoidable  emergencies.  The  second,  preparedness, 
involves  activities  by  governments,  organizations  and  individuals  to  develop 
plans  to  save  lives  and  minimize  damage.  Preparedness  measures  also  seek  to 
enhance  response  operations.  The  third,  response,  includes  activities  to 
provide  warning,  population  protection,  and  emergency  assistance.  They  also 
seek  to  reduce  the  probability  of  secondary  damage  and  to  speed  recovery 
operations.  The  fourth,  recovery,  consists  of  both  short-term  and  long-term 
activities.  Short-term  recovery  returns  vital  life-support  systems  to  minimum 
operating  standards.  Long-term  recovery  may  continue  for  a number  of  years 
after  a disaster.  Their  purpose  is  to  return  life  to  normal,  or  improved 
levels . 

Associated  with  each  policy  issue  is  a series  of  general  measures  which 
cut  across  all  emergencies  and  a series  of  emergency-specific  measures.  The 
general  measures  associated  with  each  policy  issue  are  summarized  in  Table 

1.1. 


Emergencies  may  be  classified  according  to  type  and  size.  The  types  of 
emergencies  include:  (1)  natural  disasters;  (2)  technological  emergencies; 

(3)  resource  shortages;  (4)  disorders;  and  (5)  war.  The  various  types  of 
emergencies  can  be  ranked  by  magnitude  into  local,  regional,  national,  and 
international  emergencies.  It  should  be  recognized,  however,  that  there  are 
variations  in  individual  emergencies  and  that  local  emergencies  may  be  of 
national  or  international  interest  and/or  impact. 

A framework  for  mobilizing  the  resources  which  may  be  needed  is  outlined 
in  National  Security  Decision  Directive  47  and  Executive  Order  11490  in  order 
to  address  the  types  of  emergencies  just  mentioned.  Seven  categories  serve  to 
define  the  context  of  emergency  mobilization.  These  categories  are  described 
in  Table  1.2. 

National  Security  Decision  Directive  47  states  that  it  is  the  policy  of 
the  United  States  to  have  an  emergency  mobilization  capability  that  will 
ensure  that  government,  at  all  levels,  in  partnership  with  the  private  sector 
and  the  American  people,  can  respond  decisively  and  effectively  to  any  major 
national  emergency. 
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Executive  Order  11490,  Assignment  of  Emergency  Preparedness  Functions  to 
Federal  Departments  and  Agencies,  requires  departments  and  agencies  to  prepare 
national  plans  and  programs  and  to  attain  an  appropriate  level  of  readiness 
with  regard  to  the  functions  assigned.  It  also  requires  the  Director  of  FEMA 
to  establish  Federal  policies  for  and  coordinate  all  emergency  preparedness 
activities  and  functions  of  the  Federal  Government,  and  be  responsible  for  the 
preparation  of  guidance  to  Federal  departments  and  agencies  to  assist  them  in 
performing  their  assigned  emergency  functions. 

The  final  consideration  in  emergency  mobilization  preparedness  is  the 
timing  of  management  actions.  Four  phases  have  been  established  for  planning 
purposes.  In  a real  emergency  the  phases  may  not  exist  in  clearly  discernible 
stages,  but  they  do  provide  a useful  framework  for  planning  for  the  full  range 
of  management  actions.  The  relationship  between  the  policy  issues,  the  scale 
of  the  emergency  and  the  time  frame  is  illustrated  in  Figure  1.2. 

There  exists  a diversity  of  activities  and  related  problems  that  FEMA 
must  address  in  carrying  out  its  mission.  FEMA's  activities  include 
day-to-day  data  gathering,  responding  to  the  immediate  needs  of  an  emergency, 
and  strategic  planning  and  analysis.  It  is  not  our  purpose  to  be  complete 
here,  but  instead  to  emphasize  the  decision  making  requirements  faced  by  FEMA 
and  related  organizations  and  individuals. 

Central  to  all  of  FEMA’s  activities  is  a viable  and  current  information 
support  system  that  is  directed  toward  the  needs  of  emergency  managers.  Such 
a system  is  defined  to  include  not  only  data  and  descriptive  information,  but 
also  analysis  and  interpretive  programs.  The  establishment  and  maintenance  of 
this  support  system,  although  a problem  in  itself,  cannot  be  accomplished 
without  recognition  of  the  decision  problems  and  other  related  problems  of 
emergency  managers.  These  include  the  development  and  implementation  of 
training  procedures  for  emergency  managers  at  all  levels,  the  requirement  of 
emergency  managers  at  all  levels  to  be  able  to  find  and  interpret  current 
emergency  response  rules  and  regulations,  the  need  of  FEMA  to  be  able  to 
evaluate  proposed  emergency  guidelines  and  legislation,  and  the  basic 
management  procedural  requirements  for  coordinating  multi-regional  responses 
and  allocations  of  resources.  Thus,  to  be  effective,  an  information  support 
system  must  be  designed  to  reflect  the  needs  of  these  varied  problems.  In 
particular,  one  must  be  cognizant  of  the  many  confounding  and  limiting  aspects 
inherent  in  the  broad  spectrum  of  emergency  operations.  These  include  the 
varied  nature  of  emergencies  and  the  extremely  wide  range  of  possible 
responses  requiring  formal  evaluation  procedures. 

The  time  taken  to  respond  to  an  emergency  can  also  be  critical.  Public 
safety  organizations  (e.g.,  police,  fire,  ambulance  service)  are  always  on 
duty  and  take  pride  in  their  short  response  times.  But  for  broader  emergency 
situations,  like  toxic  and  explosive  spills  or  hurricanes  that  require 
decision  structures  and  response  systems  that  are  basically  dormant,  the 
ability  to  mount  a proper  and  timely  response  is  difficult.  Here,  emergency 
managers  must  be  able  to  couple  their  static  personnel  and  resource  base  with 
sporadic  incoming  information  to  make  a dynamic  set  of  responses.  This  can 
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Table  1.1  General  Measures  Used  as  a Function  of  th  Policy  Issue 


Issue 

Measure 

Mitigation 

Building  codes 

Zoning  and  land  use  management 
Compliance  and  enforcement 
Public  education 

Preparedness 

Preparedness  plans 
Emergency  exercises/ training 
Emergency  communications  systems 
Evacuation  plans  and  training 

Response 

Activate  public  warning 
Mobilize  emergency  personnel/equipment 
Emergency  medical  assistance 
Man  emergency  operations  centers 

Recovery 

Damage  insurance/loans  and  grants 
Temporary  housing 
Disaster  unemployment  insurance 
Economic  impact  studies 
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Table  1.2  Mobilization  Categories 


Category 


Definition 


Military  Mobilization 


Industrial  Mobilization 


Economic  Mobilization 


The  act  of  preparing  for  war  or  other 
emergencies  through  assembling  and  organizing 
military  resources. 

The  process  of  marshalling  the  industrial 
sector  to  produce  goods  and  services  required 
to  support  military  operations  and  the  needs 
of  the  civil  sector  during  domestic  or 
national  security  emergencies. 

The  process  of  marshalling  the  money,  credit, 
and  taxes  needed:  (1)  to  finance  the  manage- 
ment of  the  emergency;  (2)  to  maintain  a 
stable  economy;  and  (3)  to  stimulate  key 
sectors  of  the  economy. 


Infrastructure  Mobilization  The  process  of  marshalling  the  output  of 

infrastructure  systems  to  support  the  entire 
mobilization. 


Human  Resources  Mobilization  The  process  of  marshalling  people  to  provide 

needed  labor. 


Government  Mobilization  The  process  of  marshalling  resources  of 

Federal,  State  and  local  governments  to  carry 
out  the  tasks  required  to  manage  emergencies 


Civil  Preparedness  Mobilization  The  process  of  marshalling  resources  to 

provide  protection  for  the  people,  industry, 
and  institutions  of  the  United  States  against 
the  effects  of  the  spectrum  of  emergencies. 
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Figure  1.2  Conceptual  Framework  for  Emergency  Mobilization  Preparedness 


TIME 

FRAME 


POLICY  ISSUES 


only  be  effective  if  a full  range  of  contingency  plans  has  been  developed  and 
tested  by  the  emergency  management  team.  Such  contingency  planning  is  not  a 
priority  item  at  most  levels  of  emergency  management.  It  is  costly,  it 
requires  experienced  personnel,  but  it  is  a tactical  activity  that  must  be 
placed  on  a solid  base. 

The  analysis  of  emergencies  and  emergency  responses  is  backed  up  with 
some  theory  on  the  physics  of  particular  emergencies,  but  little  theory  on  the 
behavioral  aspects  of  most  emergencies.  There  are  fire  spread  models,  nuclear 
blast  and  fallout  models,  network  flow  models,  epidemiological  models,  some  of 
which  can  be  of  assistance  in  contingency  planning,  as  well  as  in  an 
operational  situation.  A key  question  is  how  available  theory  can  be  utilized 
in  an  emergency  information  support  system,  especially  at  the  local  level. 

One  is  quick  to  recognize  that  the  major  concerns  of  emergency  management 
operations,  like  all  complex  decision  situations,  have  both  technological  and 
human  components.  And,  as  the  problem  is  of  national  importance  and  something 
must  be  done  to  understand  better  all  aspects  of  emergency  management,  there 
is  a danger  of  rushing  to  apply  technical  methodologies.  An  integrated 
emergency  information-communications  system  will  be  ineffective  if  the 
personnel  and  behavioral  aspects  of  who  is  to  use  it,  how  will  the  users  be 
trained,  and  what  the  decision  and  resource  needs  of  the  users  are  ignored. 

From  a general  emergency  perspective,  the  individuals,  and  groups  that 
concern  themselves  with  an  emergency  (i.e.,  the  stakeholders  and  actors) 
include  persons  at  the  Federal,  State  and  local  levels,  as  well  as  persons  in 
private  sector  organizations.  Thus,  emergency  management  operations  and 
supporting  systems  must  be  responsive  to  a wide  diversity  of  needs  and 
priorities . 
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PART  I 

GUIDELINES  FOR  EVALUATING  EMERGENCY 
MANAGEMENT  MODELS  AND 
DATA  BASES 
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2.  DESCRIPTIVE  ANALYSIS 


A descriptive  analysis  focuses  on  providing  background  information  on  the 
model.  Such  information  includes  the  motivation  for  the  model,  its 
organization  and  theoretical  underpinnings,  and  the  conditions  under  which  it 
was  developed. 

As  a first  step,  the  descriptive  analysis  should  document  the  policy 
context  and  the  real  world  conditions  for  which  predictions  are  desired.  This 
information  should  provide  a background  for  evaluating  the  appropriateness  and 
relevancy  of  the  model.  Information  on  the  model's  organization  promotes  a more 
thorough  understanding  of  the  model's  structure,  its  inputs  and  outputs  and 
general  flow  of  information.  A major  component  of  the  descriptive  analysis  is  a 
review  and  summary  of  key  publications  describing  the  model's  objectives, 
structure  and  principal  results.  The  literature  review  is  also  helpful  in 
evaluating  the  appropriateness  of  the  model's  structure  for  dealing  with  policy 
issues  and  the  plausibility  of  its  results.  Finally,  it  is  useful  to  know  the 
conditions  under  which  the  model  was  developed.  This  information  is  intended  to  , 
alert  the  users  or  evaluators  to  special  circumstances  of  development  likely  to 
make  the  model  either  more  or  less  suitable  for  the  proposed  application, 

2.1  MOTIVATION  FOR  THE  MODEL 


This  portion  of  the  evaluation  consists  of  four  components.  The  first 
component  focuses  on  a description  of  what  is  to  be  described  by  the  model  and 
why.  The  second  discusses  how  to  apply  the  model  to  the  problem(s)  of  interest. 
The  third  describes  how  the  various  parts  of  the  model  fit  together.  The  final 
component  provides  a means  for  documenting  which,  if  any,  previous  models  may 
have  influenced  the  development  of  the  current  model. 

To  facilitate  the  discussion  which  follows,  we  may  note  that  there  is  a 
real  world  problem  which  is  of  interest.  This  problem  provides  the  motivation 
for  model  development,  because  a model  provides  a method  of  analysis.  However, 
in  order  to  develop  a model,  it  is  necessary  to  make  certain  assumptions  and 
simplifications.  These  notions  may  be  quite  general  and  hence  serve  only  as  an 
outline  of  how  to  build  the  model;  they  may  be  thought  of  as  constituting  a 
conceptual  model.  The  four  components  of  this  portion  of  the  evaluation  define 
the  linkage  between  the  real-world  problem  and  the  conceptual  model. 

As  a first  step  we  must  document  what  information  is  available  on  the 
real-world  problem  which  must  be  analyzed  and  the  policy  and/or  other  options 
which  may  be  applied  to  affect  an  outcome.  This  step  serves  to  document  the 
policy  context  in  which  the  conceptual  model  is  to  operate.  At  the  most  basic 
level,  the  conceptual  model  would  focus  on  one  or  more  of  the  four  policy  issues 
with  which  emergency  managers  are  concerned.  Specifying  the  basic  policy  focus 
defines  which  issues  are  important  and  how  they  relate  to  the  real-world 
problem. 

The  identification  of  specific  types  of  policies  is  the  objective  of  the 
second  component.  The  way  in  which  specific  policies  are  used  to  address  the 
problem  of  interest  provides  insight  and  guidance  into  how  one  would  apply  the 
conceptual  model.  The  reviews  in  this  area  seek  to  identify  the  basic  analysis 
strategy  and  restrictions  which  may  affect  the  model's  range  of  application. 

This  review  provides  a basis  for  identifying  those  resources  which  may  be  used 
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or  are  required  to  manage  the  real-world  problem.  Special  emphasis  should  be 
placed  on  relating  these  resources  to  the  seven  mobilization  categories  used  by 
FEMA. 

The  third  component  of  the  analysis  provides  the  basis  for  a description  of  the 
overall  system.  Here  the  evaluator  documents  the  general  decision  problem  and 
outlines  how  the  various  parts  of  the  model  fit  together.  For  example,  the 
general  problem  may  fit  into  one  of  two  broad  areas,  physical  or  policy. ^ 
Additional  refinement  is  accomplished  by  identifying  the  analytical/ technical 
approach  used  in  the  model.  Researchers  in  applied  mathematics  usually  refer  to 
the  possible  approach  as  either  descriptive  or  prescriptive.  In  a descriptive 
approach,  the  solution  provided  by  the  model  aims  at  explaining  what  is 
happening  not  why  it  is  happening.  In  essence,  the  model  provides  a snapshot. 

In  a prescriptive  approach,  the  solution  provided  by  the  model  prescribes  the 
best  way  to  utilize  existing  resources.  Such  models  explain  what  should  be 
done;  they  do  not  necessarily  provide  a great  deal  of  insight  into  what  is 
happening.  An  example  of  a descriptive  approach  is  a discrete  event  simulation 
where  certain  components  in  a network  are  being  modeled  during  an  emergency 
situation.  As  the  emergency  unfolds,  certain  components  of  the  network  function 
at  reduced  levels  of  performance.  Performance  degradation  may  cause  bottlenecks 
to  develop  in  the  network;  if  serious  enough,  they  could  compromise  the  entire 
operation.  A simulation  model  enables  the  analyst  to  identify  the  likely 
bottlenecks  prior  to  an  actual  emergency.  The  basic  design  of  the  network  may 
be  changed  to  incorporate  this  information.  Although  the  design  of  the  new 
network  may  be  a distinct  improvement  over  the  old  one,  it  need  not  be  the 
optimal  configuration.  To  design  an  optimal  network  it  would  be  necessary  to 
make  use  of  specialized  optimization  techniques.  Optimization  techniques  which 
are  in  wide  use  include  linear,  non-linear,  and  dynamic  programming. 

The  final  component  provides  a means  for  documenting  how  the  model  under 
analysis  may  have  benefitted  from  previous  research.  The  objective  of  this  step 
is  to  see  how  experience  gained  from  other  modeling  efforts  may  have  led  to  the 
choice  of  particular  algorithms  or  structural  properties.  This  step  serves  to 
sharpen  the  focus  of  the  evaluation  by  highlighting  the  strengths  or  weaknesses 
of  alternative  model  specifications  brought  out  in  prior  studies. 

2.2  ORGANIZATION  OF  THE  SUBJECT 


This  portion  of  the  analysis  seeks  to  develop  an  outline  of  the  conceptual 
model.  This  is  done  by  first  cross  referencing  the  purpose  of  the  model  with 
the  basic  policy  issues  (mitigation,  preparedness,  response  and  recovery)  and 
various  emergency-specific  measures.  Once  the  relevant  policy  issues  and 
emergencies  have  been  identified,  it  is  necessary  to  define  the  level  of  detail 
of  the  model’s  output.  This  entails  a critical  review  of  the  relevant  policy 
considerations,  as  well  as  empirical  considerations  and  mode  1 ing/ sys tern 
constraints . 


^-It  is  possible  for  some  models  to  be  hybrids,  having  major  components  within 
each  of  the  two  areas. 
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There  are  three  additional  steps  in  the  analysis.  These  steps  focus  on: 

(1)  software  configuration;  (2)  variable  selection;  and  (3)  causal  and  dynamic  ' 
relationships.  Since  these  steps  are  closely  related  to  a number  of  activities 
which  are  carried  out  under  the  in-depth  review  of  the  documentation,  they  will 
be  discussed  here  only  briefly.  Readers  wishing  a more  detailed  discussion  are 
referred  to  Chapter  3. 

In  developing  a configuration  for  the  software,  it  is  necessary  to  identify 
the  major  elements  of  the  programming  approach.  This  requires  the  development 
of  a gross-level  flowchart  which  shows  all  major  modules  and  their  relationships 
to  one  another.  The  next  step  is  to  determine  what  information  is  required  to 
study  the  problem  and  why.  This  would  involve  both  intra-  and  inter-module 
transfers  of  information.  More  specifically,  the  criteria  for  selecting 
variables  should  be  stated.  Variables  should  also  be  classified  by  type  (e.g., 
exogeneous,  endogenous,  controlled)*  The  final  step  is  to  identify  the 
determinants  of  logic  flow  and  information  processing.  Since  causal  and  dynamic 
relationships  may  drive  calculations  both  within  a model  or  serve  to  define  a 
hierarchy  among  modules,  it  is  important  that  both  the  criteria  used  to  specify 
these  relationships  and  their  effect  on  the  solutions  provided  (e.g., 
convergence)  be  discussed  as  an  integral  part  of  the  conceptual  model. 

It  should  be  clear  from  the  previous  discussion  that  the  conceptual  model 
will  exert  a substantial  influence  on  both  the  mathematical  model  and  the 
operational  model.  This  influence  is  discussed  in  some  detail  in  the  section 
which  follows. 

2.3  THEORETICAL  UNDERPINNINGS 


This  portion  of  the  evaluation  has  two  components.  The  first  is  an 
analysis  of  the  economic  and  other  technical  foundations  of  the  model.  The 
second  deals  with  the  mathematical  characterization  of  the  model,  including  such 
properties  of  the  solution  algorithm  as  uniqueness,  convergence,  and  the 
behavior  of  successive  iterations  as  the  solution  is  approached. 

The  first  step  in  the  analysis  includes  the  process  by  which  the  precise 
mathematical  set  of  equations  is  deduced  from  the  conceptual  model.  An  in-depth 
analysis  is  required,  including  a determination  of  the  theoretical  basis  and 
structure  of  the  model.  A study  of  basic  technological  and  behavioral 
assumptions  made  or  implied  is  also  required.  The  analysis  must  also  consider 
whether  the  level  of  aggregation  of  activities  is  appropriate,  whether 
activities  are  insufficiently  modeled,  or  whether  activities  that  have  only 
marginal  significance  introduce  unnecessary  cost  and/or  complexity. 

The  mathematical  characterization  portion  of  the  analysis  focuses  on 
developing  a thorough  understanding  of  the  theoretical  foundations  of  the 
problem;  including  references  to  the  open  literature  where  appropriate.  It 
should  define  the  problems  solved,  describe  the  mathematical  model  employed,  and 
document  the  computational  algorithms  and  numerical  techniques  implemented  in 
the  code.  For  example,  cases  where  locally  convergent  procedures  are  used 
should  be  carefully  documented.  This  is  because  locally  convergent  procedures 
are  only  designed  to  find  a solution  near  some  initial  vector  of  values  used  to 
start  the  iteration  procedure.  It  is  important  here  to  recognize  that  local 
convergence  is  being  sought  and  that  the  initial  values  of  the  input  vector  are 
crucial  to  the  practical  success  of  the  algorithm. 
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This  portion  of  the  analysis  should  also  include  a comprehensive 
description  both  of  the  problem  solved  and  of  the  data  processing  functions 
performed.  The  description  of  the  physical  theory  in  terms  of  a mathematical 
model  should  be  reasonably  self  contained.  Sources  for  the  model  and  the 
mathematical  formulations  should  be  referenced.  Sufficient  detail  is  needed  to 
judge  the  suitability  of  the  model  for  application  to  a particular  situation. 
Assumptions  should  be  noted  and  information  given  about  limitations. 

The  relationships  between  the  two  major  components  of  this  activity  and  the 
activities  described  in  Sections  2.1  and  2.2  can  best  be  understood  through 
reference  to  Figure  2.1.  This  figure  shows  that  there  are  three  distinct  types 
of  models  which  are  derived  from  and  related  to  the  real-world  problem.  These 
model  types  are:  (1)  the  conceptual  model;  (2)  the  mathematical  model;  and  (3) 
the  operational  model.  As  indicated  before,  the  real-world  problem  and 
associated  policy  issues  result  in  the  formulation  of  a conceptual  model.  The 
purpose  of  the  conceptual  model  is  to  describe  what  is  to  be  studied  and  why. 

The  conceptual  model  may  be  based  in  part  on  a comparative  analysis  of  the 
strengths  and  weaknesses  of  other  models  in  this  problem  area  or  on  certain 
historical  information.  The  mathematical  model  makes  explicit  all  of  the 
assumptions  of  the  conceptual  model;  it  also  specifies  the  data  and  domain  of 
definition,  statistical  properties,  and  equation  structure.  The  operational 
model  is  the  version  of  the  mathematical  model  which  resides  on  the  computer 
system;  it  is  the  version  of  the  model  which  is  executed  on  the  computer  to 
address  specific  policy  and/or  related  issues. 

We  can  now  relate  the  activities  of  the  descriptive  analysis  portion  of  the 
evaluation  to  Figure  2.1.  The  methodology  described  in  Section  2.1,  motivation 
for  the  model,  is  concerned  with  the  relationship  of  the  real-world  problem  to 
the  conceptual  model.  The  methodology  described  in  Section  2.2,  organization  of 
the  subject,  serves  to  define  the  conceptual  model.  The  economic/ technical 
foundations  portion  of  the  analysis  documents  the  correspondence  between  the 
conceptual  model  and  the  mathematical  model.  The  mathematical  characterization 
portion  of  the  analysis  serves  to  define  the  mathematical  model  (e.g., 
assumptions,  data,  algorithms,  equation  structure).  The  activities  which 
comprise  the  descriptive  analysis  portion  of  the  evaluation  begin  at  the  top  of 
Figure  2.1,  move  through  the  conceptual  model  to  the  box  labeled  mathematical 
model . 

Figure  2.1  also  relates  the  techniques  used  in  a descriptive  analysis  to 
the  terms  verification  and  validation  introduced  in  Chapter  1.  Through 
reference  to  the  figure  we  can  see  that  verification  is  the  process  of 
documenting  the  relationship  between  the  mathematical  model  and  the  operational 
model.  Descriptive  analysis  sets  the  stage  for  what  follows  by  outlining  the 
linkage  between  the  real-world  problem,  the  conceptual  model  and  the 
mathematical  model.  This  linkage  should  be  carefully  explained  in  the  model's 
documentation  reports.  Both  types  of  validation,  subjective  and  objective,  are 
shown  on  the  figure.  Since  the  conceptual  model  is  an  abstraction  of  the 
real-world  problem,  subjective  validity  implies  that  the  model  satisfies  both 
its  intended  use  (as  set  forth  in  the  conceptual  model)  and  a particular  set  of 
user  requirements.  The  ideal  case,  objective  validity,  implies  that  the  model 
closely  tracks  the  real-world  problem.  Both  types  of  validation  are  shown  as 
dashed  lines  to  highlight  the  complexities  inherent  in  "validating"  emergency 
management  models. 
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Figure  2.1  Interrelationships  Between  Model  Evaluation  Activities 


L 8 


2.4  MODEL  DEVELOPMENT  CONTEXT 


The  final  design  of  a model  is  often  influenced  by  the  practical 
constraints  of  resources  and  schedules.  It  is  useful,  therefore,  to  examine  the 
administrative  conditions  under  which  a model  was  developed  and  to  use  this 
information  to  alert  the  user  or  evaluator  to  special  features  of  the  model 
which  impact  on  its  application  or  its  role  in  a system  of  models.  The 
principal  constraints  which  are  of  interest  are  resource  limitations  (project 
scheduling  and  the  availability  of  staff)  and  operational  limitations. 

Resource  limitations  must  be  evaluated  in  terras  of  how  they  impacted  the 
development  of  the  model,  especially  the  time  available  for  theoretical 
considerations.  Resource  limitations  may  result  in  design  freezes,  and 
concessions,  and  compromises  or  trade-offs  by  the  model  developers  on  its 
complexity.  The  development  of  a knowledge  base  for  a model  evaluation  should 
identify  these  shortcomings. 

Operational  limitations  which  may  affect  the  design  of  the  model  include 
the  computer  configuration,  software  support,  program  language,  portability 
(i.e.  the  ability  to  use  the  program  on  computers  having  compatible  languages), 
operational  features,  turnaround  time,  and  model-to-computer  coordination. 

The  major  sources  of  information  about  constraints  on  model  development  are 
from  documentation.  Ideally,  the  documentation  would  include  descriptions  that 
portray  the  accommodations  made  by  the  developers  of  the  model  due  to  resource 
or  operational  constraints.  Departures  from  specifications  defined  in  the 
contract  for  the  original  development  and  from  the  description  and  execution  of 
the  acceptance  test  of  the  model,  which  was  used  to  prove  compliance  with 
specifications  and  to  test  the  model's  performance  and  general  integrity,  should 
also  be  documented.  Documentation  is  usually  the  main  model  component  that  is 
impacted  by  resource  limitations. 

The  approach  for  assessing  the  features  of  model  development,  as  they 
relate  to  the  evaluation  of  the  model  in  the  context  of  user  requirements, 
consists  of  extracting  from  the  information  sources  described  above,  the 
assumptions,  compromises,  or  trade-offs  which  resulted  from  resource  or 
operational  constraints,  and  bringing  this  information  into  the  evaluation 
process  as  described  in  the  guidelines  for  the  model  evaluation  procedure. 

The  information  presented  in  this  section  touches  on  all  phases  of  the 
software  life  cycle.  Although  model  evaluation  is  carried  out  with  regard  to  a 
fixed  frame  of  reference,  recommendations  are  available  for  the  planning  and 
development  of  new  models.  One  such  set  of  recommendations  is  summarized  in 
Appendix  A. 
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3.  PROGRAM  VERIFICATION  AND  ANALYSIS 


This  component  of  the  evaluation  guidelines  focuses  on  the  clarity, 
completeness  and  adequacy  of  the  model's  documentation.  This  involves  three 
interdependent  activities:  (1)  an  in-depth  review  of  the  documentation;  (2)  a 

review  of  the  source  code;  and  (3)  an  analysis  of  experiences  in  executing  the 
model . 

The  in-depth  review  of  the  documentation  provides  the  insights  which  are 
necessary  to  perform  the  remaining  portions  of  the  evaluation.  Beyond  the  use 
of  the  documentation  as  a base  of  technical  knowledge  is  the  issue  of 
completeness.^  This  issue  is  related  to  the  recommendations  for  program 
documentation  specified  in  the  Federal  Information  Processing  Standards  (FIPS) 
Publication  38^  and  the  NBS  Special  Publication  500-73. ^ 

The  next  activity  is  to  review  the  source  code  itself.  Any  discrepancies 
between  the  documentation  and  the  source  code  should  be  recorded,  especially 
with  regard  to  their  effect  on  the  structure  of  the  program.  This  review  should 
use  the  latest  techniques  of  software  analysis  and  code  fault  isolation. 

The  analysis  of  experiences  in  executing  the  model  consists  of  several 
activities.  First,  any  problems  in  setting  up  the  model  on  the  host  system  are 
documented.  This  step  involves  interviewing  model  users  and  operating  system 
support  personnel.  Second,  the  users'  or  evaluators'  experiences  in  executing 
the  benchmark  problem^  provided  by  the  model  developers  are  recorded.  The 
benchmark  problem  helps  the  user/evaluator  to  determine  if  the  input  data  are 
properly  communicated  throughout  the  various  logical  paths  of  the  program. 
Finally,  it  is  essential  that  the  evaluators  document  their  hands-on  experiences 
with  the  model.  This  step  documents  instances  where  proper  input  data  are  used 
but  errors  are  introduced  through  improper  data  processing.  This  step  also 
documents  experiences  in  reproducing  published  results  and  the  ease  with  which 
special  purpose  modules  can  be  used  or  generated. 

3 . 1 IN-DEPTH  REVIEW  OF  DOCUMENTATION 

It  is  generally  accepted  that  the  evaluator  needs  more  to  work  with  than 
the  code  which  defines  the  operational  model.  Current  consensus  among 
authorities  states  that  proper  documentation  is  an  absolutely  essential 
ingredient  for  the  acceptability  and  use  of  any  complex  computer  model. 


*-Gass”7  S.I~  "Documenting  a Computer-Based  Model,"  Interfaces , Vol.  14,  No.  3, 
1984,  pp.  84-93. 

^Guidelines  for  Documentation  of  Computer  Programs  and  Automated  Data  Systems . 
Gaithersburg,  MD:  National  Bureau  of  Standards,  FIPS  PUB  38,  1976. 

^Computer  Model  Documentation  Guide.  Gaithersburg,  MD:  National  Bureau  of 
Standards,  Special  Publication  500-73,  1981. 

4A  benchmark  should  be  a well-defined  problem  and  solution  chosen  by  the 
developer  to  exercise  a large  portion  of  the  model’s  logic.  This  problem 
should  serve  as  an  important  vehicle  for  determining  if  the  model  is  running 
properly  on  the  host  system. 
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A substantial  body  of  documentation  is  required  if  a model  evaluation  is  to 
be  effective.  Researchers  at  the  National  Bureau  of  Standards  and  elsewhere 
have  identified  four  basic  types  of  documentation  reports  which  are  essential 
for  efficient  use  of  a model.  These  reports  are:  (1)  Management  Summary  Manual; 
(2)  User’s  Manual;  (3)  Programmer’s  Manual;  and  (4)  Analyst’s  Manual.  These 
reports  are  extremely  important  in  any  comprehensive  model  evaluation.  A brief 
description  of  each  report  is  therefore  given  in  the  text  which  follows  to 
promote  a more  complete  understanding  of  the  activities  involved  in  an  in-depth 
review  of  the  documentation. 

The  first  manual  is  designed  as  a management  tool.  It  provides  the 
information  necessary  to  assess  the  model’s  input  requirements  (including  time, 
money,  and  other  resources)  and  the  usefulness  of  the  model's  results.  The 
Management  Summary  Manual  focuses  on  how  the  model  can  facilitate  the  decision 
making  process  rather  than  the  specifics  of  how  to  set  up  and  run  the  model. 

The  second  manual  is  designed  as  a reference  document  for  a nonprogramming 
model  user.  Information  contained  in  this  manual  is  similar  to  the  first  but 
with  increased  emphasis  on  detail.  In-depth  discussions  of  the  following  topics 
are  included:  the  model's  logical  structure;  the  input  data  requirements;  the 

results  produced  by  the  model;  and  the  use  of  the  model  results. 

The  third  and  fourth  documents  are  designed  for  use  by  programmers  and 
analysts,  respectively.  The  third  manual  provides  guidelines  for  maintaining 
and  modifying  the  model.  These  guidelines  should  be  of  sufficient  detail  to 
enable  the  programmer  to  understand  the  operation  of  the  model  and  to  trace 
through  it  for  debugging,  for  making  modifications,  and  for  determining  if  and 
how  the  model  can  be  converted  to  other  computer  systems.  The  fourth  manual 
differs  from  the  third  in  that  its  emphasis  is  on  the  model's  functional 
structure,  types  of  algorithms  used,  and  the  techniques  employed  for  model 
verification  and  validation. 

The  first  step  in  this  activity  is  to  review  the  documentation  both  for  its 
completeness  and  its  clarity.  This  step  greatly  facilitates  the  comparison  of 
the  model's  purpose  and  purported  capabilities  with  the  user's  requirements. 
Specific  tasks  within  this  activity  are  a determination  of  what  the  model  is  to 
do  and  its  intended  range  of  application.  Issues  at  stake  within  this  part  of 
the  review  are  the  motivation  and  objectives  of  the  model,  its  scope  and 
approach,  and  its  assumptions  and  restrictions. 

The  document  or  documents  which  describe  the  theory  should  detail  specific 
information  about  the  model  being  evaluated.  The  main  goal  here  is  to  detail 
the  mathematical  methods  used  and  their  relationships  to  the  real-world  problem. 
The  algorithms  used  to  obtain  numerical  solutions  from  a set  of  relationships 
(e.g.,  equations)  should  be  described  with  references  to  particular  algorithms 
and  numerical  techniques  provided. 

The  precision  of  results  obtained  by  important  algorithms  and  any  known 
dependence  on  particular  types  of  computers  or  operating  systems  should  be 
described.  For  iterative  solutions,  the  use  and  interpretation  of  convergence 
tests  and  recommended  values  of  convergence  criteria  should  be  included.  For 
probabilistic  solutions,  the  precision  of  results  should  be  discussed. 
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A related  task  is  to  develop  a precise  understanding  of  how  the  model 
works.  This  entails  an  analysis  of  the  logical  flow  of  data  through  the  model 
from  the  entry  of  input  data  to  the  generation  of  the  output.  Steps  should  also 
be  taken  to  relate  the  model  elements  and  data  flow  to  the  real-world  system 
(e.g.,  physical  elements  and  information  flows).  Although  particular  process 
equations  may  be  described,  documentation  providing  the  rationale  for  selecting 
a particular  model  input  or  parameter  may  be  missing.  Because  major 
uncertainties  can  arise  from  uncertainties  in  specifying  the  input  data,  the 
process  for  selecting  the  input  must  be  carefully  documented  and  understood. 

Although  an  analysis  of  the  input  data  is  the  subject  of  Chapter  4 (Data 
Audit),  it  is  important  at  the  earliest  stage  to  characterize  the  overall  input 
data  structure  and  the  data  media.  Specifically,  for  each  input  variable  one 
should  give:  (1)  the  variable  name;  (2)  a description  and  definition;  (3)  the 

source;  (4)  the  procedure  through  which  the  data  was  collected  or  generated;  (3) 
dimensional  units  of  the  variable;  and  (6)  formats  for  reading  the  information. 
It  is  also  desirable  to  provide  information  on  if  or  how  specific  data  entries 
can  be  updated  or,  if  appropriate,  ranges  and/or  default  values  which  may  be  set 
internally. 

The  interface  between  the  model  and  the  relevant  data  files  must  also  be 
covered  in  the  documentation.  Details  which  specify  the  names,  usage  (input, 
output,  scratch)  and  structure  of  external  and  temporary  data  files  should  be 
provided.  This  includes  a discussion  of  program  procedures  related  to  the  use 
and  maintenance  of  data  files. 

As  the  review  of  the  documentation  progresses,  the  evaluator  should  focus 
on  the  development  of  a complete  technical  description  of  the  model  with 
particular  emphasis  on  its  operational  details.  The  documentation  reports  which 
focus  on  a description  of  the  source  program  should  provide  the  basis  for  this 
activity.  These  reports  should  include:  (1)  the  identification  of  the  source 
language  (e.g.,  FORTRAN  77);  (2)  references  to  any  relevant  software  standards 
(e.g.,  ANSI  X3. 9-1978);  and  (3)  a flow  chart  showing  the  overall  program 
structure  and  logic,  and,  where  appropriate,  detailed  flow  charts. 

Most  complex  models  contain  certain  programming  considerations  which  are 
important  for  successful  implementation.  At  least  four  are  of  sufficient 
importance  that  they  should  be  checked  against  the  existing  documentation. 

First,  the  system  control  commands  required  to  execute  the  program  (including 
options)  must  be  clearly  explained.  Second,  the  storage  allocation  and  data 
management  procedures  should  be  described.  Special  attention  should  be  paid  to 
any  requirements  for  storage  and/or  data  manipulations  which  are  problem 
dependent.  In  such  cases,  alternatives  which  affect  data  storage  and  use  should 
be  discussed.  Third,  the  overlay  or  segmentation  scheme  for  the  model  should  be 
documented.  Finally,  any  restart,  recovery  or  successive  case  capabilities 
should  be  discussed.  This  issue  could  be  of  critial  importance  where  models  are 
used  in  the  emergency  response  mode  which  may  entail  operation  in  a hostile 
environment.  Provisions  for  graceful  degradation  of  service  and  manual 
operation  and/or  restart  should  therefore  be  carefully  explained  and  related  to 
the  real-world  problem. 
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Many  of  the  issues  raised  in  the  previous  discussion  touch  upon  the  topics 
of  usage  and  transportability.  This  part  of  the  documentation  review  seeks  to 
describe  how  the  model  is  to  be  used  and  what  steps  are  necessary  to  run  it  on 
another  computer  system.  Critical  factors  which  affect  both  the  ease  with 
which  a model  can  be  used  on  one  system  or  set  up  on  another  are:  (1)  a 

run-stream  description;  (2)  a sample  model  run;  (3)  a discussion  of  options;  (4) 
an  interpretation  of  the  model's  output;  (5)  a guide  to  troubleshooting  when 
executing  with  user-generated  data;  and  (6)  a set  of  provisions  for  software 
exchange.  At  a minimum,  the  first  four  factors  should  be  addressed  in  the 
documentation,  if  the  model  is  to  be  either  usable  or  transportable. 

The  operating  system  control  commands  (either  as  generic  statements  or  as 
card  images)  required  to  execute  the  program  should  be  given  with  an  indication 
of  any  interdependencies  with  either  input  options  or  data  files.  The  program 
output  should  be  described  with  relationships  shown.  Any  normalization  of 
results  should  be  described  and  associated  dimensional  units  listed.  Whenever 
the  model  is  transferred  from  one  system  to  another,  the  relevant  standards 
governing  information  transfer  should  be  followed.  This  would  include 
computer-readable  material  written  on  magnetic  tape  which  lists  as  separate 
files:  (1)  the  source  code;  (2)  any  auxiliary  code;  (3)  a benchmark  data  set; 

and  (4)  the  output  from  running  the  benchmark  data  set. 

3.2  REVIEW  OF  SOURCE  CODE 

This  portion  of  the  evaluation  determines  whether  the  operational  model 
runs  as  intended  by  the  developer.  More  precisely,  we  determine  if  the  source 
code  checks  against  the  mathematical  model  described  in  the  documentation 
reports.  Deficiencies  can  arise,  both  in  the  translation  from  statement  of 
purpose  to  model  formulation,  and  in  the  translation  from  the  mathematically 
stated  model  to  the  source  code.  Determining  the  equivalence  of  the  two,  or 
documenting  the  differences,  is  an  important  part  of  the  evaluation  process. 

The  source  code  is  checked  carefully  against  the  model  specification,  as 
determined  from  existing  documentation  and  augmented  where  necessary  with 
supplemental  information  provided  by  the  model  evaluator.  Critical  issues  which 
are  analyzed  include:  (1)  the  relationship  between  assumptions  as  stated  in  the 

documentation  reports  and  as  they  are  written  into  the  code;  (2)  the 
identification  of  critical  sets  of  calculations;  (3)  the  identification  of 
hard-wired  parameters  or  sections  of  logic  which  are  by-passed;  and  (4)  the 
effect  of  default  settings. 

The  input  data  must  also  be  shown  to  be  properly  communicated  throughout 
the  various  logical  paths  of  the  computer  program.  It  is  possible  for  the 
equations  to  be  properly  programmed  and  for  the  input  to  be  read  correctly,  but 
for  subsequent  difficulties  to  be  introduced  through  code  default  values  or 
improper  data  processing. 

It  is  important  to  point  out  that  the  review  of  the  source  code  should  be 
selective.  The  earlier  review  of  the  model's  documentation  should  provide 
insights  as  to  where  important  information  is  read,  transferred  or  output.  It 
is  these  areas  where  a line-by-line  review  of  the  code  should  be  focused.  This 
review  may  make  use  of  debugging  options  which  allow  for  tracing  or  for  checking 
the  values  of  arguments  in  an  array. 
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A thorough  review  of  the  source  code  should  produce  several  tangible 
benefits.  First,  it  should  classify  the  role  and  function  of  the  MAIN  program 
and  each  subprogram.  Second,  for  each  subprogram,  it  should  indicate  those 
subprograms  which  call  it  and,  in  turn,  those  subprograms  it  calls.  Third,  it 
should  provide  an  understanding  of  how  problem  variables  and  constants  relate  to 
program  mnemonics.  Fourth,  it  should  critically  examine  the  relationship 
between  the  actual  model  outputs  and  those  described  to  the  users  in  the 
documentation  reports.  This  review  may  focus  on  a variety  of  topics  ranging 
from  a check  on  formats  used  to  create  output,  to  the  effect  of  options  and 
print  switches  on  the  form  and  substance  of  the  output.  Fifth,  it  should 
identify  all  shared  storage  assignments  (e.g.,  COMMON  statements  in  FORTRAN). 
Sixth,  it  should  identify  the  series  and  the  level  of  the  operating  system, 
language  processors,  and  associated  subroutine  libraries  invoked  by  the  model. 
Finally,  it  should  describe  known  deviations  from  the  manufacturer's  supported 
software  that  are  required  by  the  model  (e.g.,  local  mathematical  and  utility 
routines,  and  other  installation-dependent  software). 

There  is  a major  benefit  to  the  model  user  associated  with  an  independent 
review  of  the  source  code.  The  evaluator,  by  cooperating  with  the  model  user 
and  developer,  can  provide  some  documentation,  if  necessary,  rather  than 
pointing  out  that  certain  key  components  are  incomplete  or  missing.  Naturally, 
the  documentation  provided  by  the  evaluator  is  not  always  fully  consistent  with 
the  views  of  the  modeler.  However,  if  the  evaluator  helps  to  provide  the 
necessary  documentation  about  the  condition  of  the  model  and  resists  the 
temptation  to  improve  or  replace  specific  features  without  the  concurrence  of 
the  modeler,  then  they  make  a contribution  that  outweighs  any  potential 
disadvantage . 

3.3  IMPLEMENTATION  OF  THE  MODEL  ON  THE  HOST  SYSTEM 

This  portion  of  the  evaluation  focuses  on  three  activities.  The  first 
activity  aims  at  developing  information  which  documents  the  model's 
operating  characteristics.  The  second  activity  seeks  to  determine  what  special 
attributes  of  the  host  system  were  built  into  the  model.  The  third  activity 
documents  information  on  the  user's  perception  of  the  model. 

Only  the  first  activity  is  traditionally  included  in  the  documentation  for 
the  model.  Unfortunately,  the  statistics  which  are  generated  on  run  times, 
execution  costs  and  core  requirements  for  the  host  system  should  be  viewed  as 
machine  dependent. ^ However,  if  the  cases  which  are  analyzed  and  used  to 
generate  these  statistics  are  carefully  chosen,  they  can  teach  the  user  a great 
deal  about  the  relative  cost  and  efficiency  of  running  the  program  on  the  host 
system. 

In  the  previous  section,  we  discussed  how  a critical  review  of  the  source 
code  could  help  to  determine  what  special  attributes  of  the  host  system  were 
built  into  the  model.  Another  source  of  information  on  this  subject  is  the 
current  set  of  users  on  the  host  system.  Interviews  with  the  users  may  produce 


^Unless  the  model  has  been  run  on  different  hardware/sof tware  configurations, 
these  statistics  may  provide  only  a rough  idea  of  what  it  would  be  like  to  run 
the  model  on  another  system. 
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information  on  their  experiences  in  replicating  the  benchmark  problem  or  of 
specific  tests  which  they  may  have  run  to  test  their  hypotheses  about  how  the 
program  works.  It  may  also  be  possible  to  review  memos  written  while  the  model 
was  being  built  which  document  time  or  funding  constraints  which  may  have  had  an 
adverse  effect  on  the  final  product  (see  Section  2.4). 

The  last  activity  seeks  to  evaluate  the  user’s  perception  of  the  model. 
Information  collected  here  is  of  particular  value  in  determining  if  the  model  is 
useful,  if  it  is  usable  on  the  host  system,  and  if  it  is  being  used  as  a 
decision-aiding  tool.  To  obtain  this  information  it  is  necessary  for  the 
evaluator  to  interview  the  users  of  the  model.  There  are  several  issues  which 
need  to  be  addressed  in  order  to  provide  a basis  for  a more  detailed  evaluation 
of  model  usability.  First,  the  evaluator  must  determine  how  the  user’s 
understanding  of  the  model  affects  their  perception.  For  example,  casual  users 
may  believe  the  model  is  too  "data  intensive"  or  "output  oriented,"  whereas  more 
frequent  users  may  have  concerns  about  how  to  combine  options  to  address  a 
specific  policy  issue.  Second,  whenever  someone  works  with  a large-scale  model 
over  an  extended  period  of  time,  they  encounter  problems.  The  source(s)  of  the 
problem(s)  may  be  due  to  user  error  or  to  faults  within  the  program.  If  it  is 
possible  to  document  some  of  the  more  common  problems  (either  real  or 
perceived),  they  may  provide  valuable  information  regarding  the  usability  of  the 
program.  Finally,  the  user’s  assessment  of  the  flexibility  of  the  model  must  be 
carefully  documented.  It  is  important  to  qualify  how  the  term  flexibility  is 
being  used  because  there  are  two  notions  at  work.  The  first  notion  of 
flexibility  deals  with  the  nature  of  emergency  management.  Emergencies  by 
definition  are  low-probability  and/or  high-loss  events.  The  model  should  be 
designed  so  that  certain  variables  (including  coefficients)  can  experience  large 
changes  in  value  and  the  model  will  still  converge.-*-  The  second  notion  of 
flexibility  deals  with  the  way  in  which  specific  policy  issues  are  addressed. 

For  example,  the  model  may  require  the  values  of  certain  inputs  to  be  preset. 

The  question  would  then  be  how  to  relate  a specific  policy  to  a particular  input 
or  set  of  inputs.  Is  the  relationship  clear?  Would  someone  else  select  the 
same  variable  or  set  of  variables? 

3.4  DOCUMENTATION  OF  EVALUATOR’ S HANDS-ON  EXPERIENCES 

• A thorough  evaluation  requires  hands-on  experience  with  the  model  in 
question  to  fully  understand  the  requirements  for  its  effective  use.  Hence, 
this  task  must  include  the  facility  and  organizational  arrangements  that  permit 
access  to  the  model  either  remotely  or  by  conversion  and  operation  on  the 
evaluator’s  computer  system.  Regardless  of  whether  the  model  is  run  on  the 
host  system  or  on  the  evaluator's  computer  system,  it  is  essential  that  the 
computer  code  be  frozen  at  some  point  in  time  so  that  the  evaluation  can  be 
conducted  with  respect  to  a fixed  frame  of  reference. 

The  first  activity  which  must  be  documented  is  the  relative  ease  with  which 
the  model  can  be  used.  This  includes  an  estimate  of  the  time  it  takes  to  learn 
enough  about  the  model  to  perform  a meaningful  analysis.  The  precise 
specification  of  the  model,  as  executed  in  terms  of  documentation,  data  and 
control  statements,  should  be  given  so  that  processing  considerations  can  be 


-*-No  assertion  is  being  made  that  the  value  to  which  the  model  converges  is 
correct.  We  only  require  here  that  the  model  produces  a solution. 
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evaluated.  The  experiences  of  the  evaluation  team  can  serve  to  further  document 
the  model’s  operating  characteristics.  For  example,  statistics  on  run  time, 
execution  costs  and  core  requirements  can  be  compared  to  any  which  may  have  been 
reported  in  the  documentation  reports. 

A related  topic  is  the  evaluator's  experience  in  obtaining  solutions  and 
replicating  published  results.  The  main  activity  here  is  to  identify  those 
model  attributes  which  promote  or  complicate  its  use  by  third  parties.  For 
example,  in  performing  a sensitivity  analysis,  it  may  be  necessary  to  construct 
a coordinated  set  of  runs.  If  the  method  of  model  sampling  is  used  (see  Chapter 
5),  the  model  is  run  according  to  a prespecified  experimental  design.  This  may 
require  special  purpose  routines.  It  is  therefore  necessary  to  determine 
whether  such  special  purpose  routines  are  easily  generated.  Specific  issues 
which  must  be  addressed  include:  (1)  modifications  to  the  model's  basic  logic 

flow;  and  (2)  any  input  and/or  output  considerations. 

A major  benefit  of  the  evaluator's  hands-on  experiences  with  the  model  is 
an  assessment  of  the  adequacy  of  the  model's  maintenance  plan.  Large  models 
which  are  used  as  inputs  to  policy  analysis  are  constantly  being  changed.  Three 
issues  are  of  crucial  importance  on  a model's  software  maintenance  plan;  they 

are:  (1)  unscheduled  updating;  (2)  periodic  updating;  and  (3)  review. 

Unfortunately,  unscheduled  updating  is  the  most  common;  it  is  also  the  least 
conducive  to  adequate  software  maintenance.  Unscheduled  updating  occurs  when 
errors  are  found  in  the  program  which  entail  code  changes.  Since  the  code 
changes  are  often  made  with  a view  towards  what  is  happening  locally,  large 
sections  of  logic  in  other  parts  of  the  program  may  inadvertently  get  altered  or 
turned  off.  Similarly,  new  program  applications  may  be  identified  causing  the 
model  to  be  pressed  into  a service  for  which  it  was  not  originally  designed.  On 
the  other  hand,  periodic  updating  is  usually  associated  with  cases  in  which  the 
entire  data  base  or  key  portions  of  the  data  base  are  revised.  It  is  also 
possible  that  empirical  relationships  will  be  reestimated,  producing  new  values 
for  key  parameters.  These  activities  may  result  in  a new  set  of  documentation 
reports  or  an  abridged  version  which  documents  the  changes  which  were  made.  For 
example,  all  major  input-output  tables  which  are  based  on  the  Department  of 
Commerce  benchmark  are  on  a five  year  updating  cycle.  The  fact  that  model 
documentation  may  be  adequate  at  one  point  in  time  does  not  imply  that  it  will 
be  adequate  at  some  future  date.  In  order  to  insure  that  the  software  is 
maintained  properly  and  the  documentation  is  kept  up-to-date,  it  is  necessary  to 
have  a formal  review  procedure.  This  procedure  should  document  both  the 
strengths  and  weaknesses  of  the  model.  The  end  result  of  the  review  is  whether 
the  model  should  continue  in  use  as  is,  be  modified,  or  not  be  used.  It  should 
be  clear  that  once  a model  has  been  subjected  to  a critical  evaluation,  a 
procedure  which  calls  for  periodic  reviews  would  be  a logical  followup. 
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4.  DATA  AUDIT 


The  confidence  which  a user  or  decision  maker  has  in  a model,  and  therefore 
its  utility,  are  increased  significantly  if  one  can  document  the  sources  of  the 
data.  It  is  also  desirable  to  show  that  the  projected  impacts  of  the  input  data 
uncertainties  on  the  calculated  responses  of  interest  are  sufficiently  small 
that  they  do  not  invalidate  the  intended  uses  of  the  model.  The  focus  of  the 
data  audit  is  therefore  on  the  accuracy  and  internal  consistency  of  the  data 
base . 


The  data  audit  portion  of  the  evaluation  requires:  (1)  review  and 

clarification  of  data  definitions;  (2)  analysis  of  the  nature  and  structure  of 
the  data  base;  (3)  determination  of  data  sources;  (4)  analysis  of  the  file 
computerization  and  accessibility  and  (5)  analysis  of  the  quality  of  selected 
important  data  by  estimating  uncertainties  as  bounds. 

Specific  techniques  which  should  be  applied  as  an  integral  part  of  the  data 
audit  are  based  both  on  exploratory  data  analysis  and  classical  statistics.  Due 
to  their  interactive  graphical  analysis  capabilities,  software  packages  such  as 
DATAPLOT^  are  especially  attractive  for  performing  a data  audit.  These 
techniques  are  essential  to  any  thorough  evaluation  because  they  facilitate  the 
analysis  of  descriptive,  relational  and  structural  properties  of  the  data  base. 
This  information  is  particularly  useful  in  analyzing  the  interactions  between  a 
particular  model  and  data  base.  Special  attention  should  be  placed  on 
identifying  and  determining  the  extent  of  any  da ta-imposed  constraints  on  model 
formulation. 

4.1  IN-DEPTH  REVIEW  OF  DOCUMENTATION 

Many  of  the  items  associated  with  an  in-depth  review  of  the  model's  data 
files  have  already  been  discussed  in  Section  3.1.  Since  the  model  and  its  data 
files  are  closely  related,  the  information  used  to  perform  a data  audit  are 
discussed  in  all  four  documentation  reports.  From  a practical  standpoint, 
however,  one  would  expect  that  more  emphasis  would  be  placed  on  data  issues  in 
the  User's  Manual  and  Programmer's  Manual.  This  is  because  these  reports  are 
concerned  with  the  everyday  running  of  the  model  and  its  maintenance  procedures, 
respectively. 

At  the  most  basic  level,  these  reports  should  describe  the  overall  input 
data  structure  and  the  data  media  (e.g.,  tape,  cards,  disks).  They  should 
include  a table  that  shows  the  input  data  file  names,  their  media,  and  any 
general  data  limitations.  The  table(s)  should  also  describe  the 
interdependencies,  if  any,  among  input  data  files. 

Specific  data  inputs  are  normally  organized  in  related  groups  or  as  data 
records  that  are  entered  on  a card  image.  These  related  groups  of  data 
establish  and  define  a data  file  and  should  be  described  together.  The  input 
data  files  and  the  items  within  each  data  file  should  be  discussed  in  the  order 
of  their  appearance  in  the  run  stream. 


^■Filliben,  J.J.  DAT AP LOT  - Introduction  and  Overview.  Gaithersburg,  MD: 
National  Bureau  of  Standards,  Special  Publication  667,  1984. 
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Any  detailed  description  of  a data  file  should  include  four  items:  (1)  the 

file's  name;  ( 2 ) the  number  of  inputs;  (3)  a list  of  related  data  files;  and  (4) 
a description  of  data  records.  The  first  item  should  give  an  overview  of  the 
data  file's  contents  and  its  purpose.  The  second  item  should  include  any 
factors  which  influence  the  total  number  of  inputs  from  the  data  file,  as  well 
as  the  maximum  number  of  records  that  may  (or  must)  be  used  to  execute  the 
model.  The  third  item  lists  any  data  files  whose  contents  depend  on  or  dictate 
the  input  values  of  the  file  under  analysis.  The  fourth  item  provides  general 
comments  on  the  format  of  the  data  records  followed  by  a description  of  each  of 
the  data  records. 

The  preferred  approach  for  summarizing  the  information  on  the  four  items 
just  described  is  through  the  use  of  a data  dictionary.  A comprehensive  data 
dictionary  consists  of  two  parts:  (1)  a dictionary  of  input  variables;  and  (2)  a 
dictionary  of  internal  variables.  Both  dictionaries  should  be  arranged  in 
alphabetical  order  by  variable  name. 

The  dictionary  of  input  variables  provides  an  in-depth  summary  of  all  input 
terms.  Each  array  dimension  should  be  defined  and,  where  applicable,  references 
should  be  made  to  the  effects  caused  by  variable  settings  in  other  parts  of  the 
input  deck..  If  it  is  necessary  to  refer  to  a specific  element  within  an  array, 
then  a labeling  convention  should  be  adopted  and  used  throughout  the  dictionary. 
A mnemonic  which  helps  associate  the  variable  with  its  purpose  should  also  be 
given.  For  example,  the  key  letters  in  the  related  name  may  be  capitalized 
and/or  underlined.  The  specification  through  which  the  variable  is  read  into 
the  program  (e.g.,  variable  type  in  a FORMAT  statement)  should  then  be  given. 

It  is  also  necessary  to  describe  where  within  the  model  the  variable  is  read. 

If  the  variable  has  a range,  the  range  should  be  given.  If  the  input  data  are 
constructed  through  a series  of  worksheets,  then  the  variable  should  be 
cross-referenced  to  the  appropriate  worksheet.  A complete  definition  of  the 
input  variable  should  then  be  given;  any  additional  information  should  also  be 
noted  at  this  time. 

The  dictionary  of  internal  variables  focuses  on  those  variables  which  are 
used  for  intermediate  calculations  and  hence  are  not  directly  under  the  control 
of  the  user.  Ideally,  four  types  of  variables  should  be  included  in  this 
dictionary:  (1)  any  variable  which  appears  in  an  output  report  but  is  not  an 

input;  (2)  any  variable  which  appears  in  a COMMON  statement;  (3)  any  variable 
which  is  passed  as  an  argument  in  a call  to  a subprogram;  and  (4)  any  variable 
which  appears  in  a DIMENSION  statement  but  is  not  otherwise  covered.  The 
discussion  in  the  dictionary  of  internal  variables  should  parallel  that  of  the 
dictionary  of  input  variables.  The  major  purpose  of  this  dictionary,  however, 
is  to  provide  enough  information  to  enable  programmers  to  make  changes  to  the 
source  code  dictated  by  either  user  needs  or  operating  system  characteristics. 

An  important  part  of  model  usability  is  data  collection.  Explicit 
instructions  for  data  collection  and  maintenance  should  therefore  be  included  in 
the  appropriate  model  documentation  reports.  These  instructions  should  include 
the  identification  of  the  parties  responsible  for  data  collection  and 
ma intena nee . 

At  the  most  basic  level,  the  documentation  reports  should  discuss  the  data 
sources  for  each  data  file.  The  discussion  should  identify  the  form  in  which 
the  raw  data  are  available,  and  if  appropriate,  other  organizational  units  from 
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which  the  data  may  be  collected.  Any  special  statistical  or  other  techniques 
used  to  obtain  or  generate  the  data  should  be  carefully  explained.  A flow  chart 
that  illustrates  the  major  data  collection  steps  and  their  sequence  is  highly 
desirable.  Updating  procedures  should  also  be  explained.  The  documentation 
reports  should  provide  step-by-step  instructions  for  maintaining  the  data  files. 
If  other  programs  are  used  to  update  the  data  files,  they  should  be  identified 
and  instructions  for  their  use  given.  As  before,  a flow  chart  that  illustrates 
the  major  updating  procedures  and  their  sequence  should  be  included. 

4.2  ANALYSIS  OF  FILE  COMPUTERIZATION  AND  ACCESSIBILITY 


A variety  of  items  should  be  checked  to  ensure  that  the  data  files  have 
been  properly  mechanized.  Some  of  the  information  on  these  items  will  be 
uncovered  during  the  review  of  the  model’s  documentation  reports  and  some  will 
be  uncovered  during  the  review  of  the  source  code.  Information  uncovered  in  the 
documentation  review  which  affects  file  mechanization  should  include:  (1)  an 

outline  of  the  general  contents  and  organization  of  the  data  files;  (2) 
references  to  other  programs  which  create,  modify  or  edit  these  files;  and  (3) 
the  relationship  of  the  data  files  to  the  execution  of  the  program.  Additional 
information  governing  special  input  techniques  and  requirements,  the  handling  of 
consecutive  cases  and  the  general  conventions  governing  default  values  should 
also  be  a natural  by-product  of  the  documentation  review.  Information  uncovered 
in  the  review  of  the  source  code  focuses  on  such  items  as:  (1)  the  dimensions 

of  data  arrays;  (2)  the  dependence  of  data  storage  requirements  on  problem  input 
parameters;  (3)  the  restrictions  on  the  range  of  values  of  variables;  (4)  the 
program's  restart  and  recovery  procedures;  and  (5)  the  program's  error  messages 
and  their  causes. 

A critical  component  in  the  analysis  of  file  computerization  is  a 
description  of  all  data  structures  internal  to  the  model.  This  description 
should  focus  on  both  local  and  global  variables,  arrays  and  data  records. 

Local  data  structures  should  contain  the  meaning  and  purpose  of  all  local 
variables,  arrays  and  data  records  (local  data  structures  have  their  values 
defined  only  within  particular  routines).  To  promote  a better  understanding  of 
the  file,  local  data  structures  should  be  associated  with  the  subprograms  in 
which  they  appear. 

Global  data  structures  should  contain  the  meaning  and  purpose  of  all 
global  variables,  arrays,  and  data  records  (global  data  structures  are  defined 
throughout  the  model). 

The  relationship  between  local  data  structures,  global  data  structures 
and  specific  data  files  can  be  understood  better  by  adopting  the  following 
convention:  (1)  general  data  are  summarized  in  tables;  (2)  organizational 

characteristics  are  summarized  in  figures;  and  (3)  the  contents  of  a specific 
file  are  listed  in  exhibits.  The  tables  focus  on  generic  information  only 
(e.g.,  variable  name,  format,  purpose,  range,  etc.).  Cross-referencing  between 
exhibits,  figures  and  tables  should  be  sufficient  to  enable  the  programmer 
and/or  evaluator  to  understand  how  the  model  approaches  an  application  problem. 

Global  data  structures  must  be  made  available  to  the  various  subprograms  of 
the  model.  The  most  frequently  used  method  is  COMMON  storage.  By  using  the 
COMMON  storage  accessing  method,  frequently  used  data  may  be  referenced  by  the 
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same  mnemonic  throughout  the  program.  The  variables  in  the  COMMON  storage  areas 
may  be  the  user's  input,  or  program  defined  data  which  may  be  character,  integer 
or  real.  No  particular  ordering  is  used  in  assigning  variables  to  the  areas. 
However,  those  areas  used  for  user's  input  are  allocated  in  the  same  order  as 
the  input  records.  The  documentation  reports  should  contain  a description  of 
COMMON  storage  areas.  These  descriptions  should  present  a synopsis  of  the  type 
of  variables  stored  in  each  area  as  well  as  a listing  of  variables  in  each  area. 
A series  of  tables  listing  the  variable  name  (including  dimensions  for  arrays), 
its  contents,  ranges  on  values,  and  data  type  (e.g.,  character,  real,  integer) 
should  be  sufficient  for  cross-referencing. 

Most  large-scale  models  produce  two  basic  types  of  outputs,  normal  outputs 
and  error  messages®  The  normal  outputs  have  been  discussed  under  several 
headings  in  Chapter  3.  Error  messages  were  mentioned  earlier;  they  are  used  to 
address  the  everyday  problems  of  incorrect  formatting,  recording  and  sequencing 
errors  in  user-generated  data.  They  are  also  useful  in  identifying 
inconsistencies  within  a data  file.  If  the  model  is  at  all  complex,  it  should 
contain  a system  for  edit-checking  the  values  of  key  input  variables.  If  an 
error  is  encountered,  a message  should  be  printed  out  to  the  user  to  help  locate 
the  source  of  the  error  and  then  correct  it.  The  model's  documentation  should 
thus  include  a description  of  the  error  messages  output  with  suitable 
cross-referencing  to  the  input  deck  and/or  data  file.  The  description  of  the 
error  message  should  include:  (1)  the  diagnostic  (i.e.,  the  error  message 

output  to  the  user);  (2)  the  action  taken  (e.g.,  program  terminates  or  said 
variable  is  set  to  its  maximum  or  minimum  allowable  value);  (3)  the  data  output 
(e.g.,  the  value  read,  the  maximum  value  allowed,  or  the  column  number  of  an 
array  undergoing  a transformation);  and  (4)  the  remedy  (e.g.,  check  the  settings 
on  read  loops,  check  the  sequencing  of  card  images,  check  for  missing  data). 

Care  should  be  taken  so  that  the  description  offers  sufficient  detail  to  enable 
the  user  to  locate  and  correct  the  error  based  on  the  prescription  given  in  the 
remedy.  Any  instances  where  error  messages  are  suppressed  due  to  settings  on 
print  switches  should  also  be  given  to  alert  the  user  that  it  may  be  necessary 
to  look  for  second-order  effects. 

4.3  SPECIALIZED  TECHNIQUES:  AN  OVERVIEW 


Specialized  techniques  for  a data  audit  may  focus  on  two  distinct  types  of 
data  analysis:  (1)  internal;  and  (2)  external.  In  an  internal  data  analysis 

quantitative  comparisons  may  be  made  across  records  for  a given  variable,  across 
variables  for  a given  record,  or  across  time  for  a given  variable.  In  an 
external  data  analysis  quantitative  comparisons  are  made  to  comparable  data  in 
other  data  bases  or  in  published  documents.  The  focus  of  this  section  is  on  an 
internal  data  analysis. 

An  internal  data  analysis  has  three  basic  objectives.  The  first,  and  most 
basic,  is  to  produce  a frozen,  archived  and  documented  data  base  ready  for 
efficient  computer  analysis.  The  reasoning  here  is  the  same  as  in  the  case  of 
program  verification  and  analysis,  namely,  the  data  audit  must  be  conducted  with 
respect  to  a fixed  frame  of  reference.  The  second  objective  is  to  produce  a 
general  understanding  of  the  data  file(s).  This  includes:  (1)  a general 
understanding  of  the  key  variables;  (2)  a rough  cut  at  response  meaningfulness 
in  terms  of  agreement  to  definition  and  consistency;  and  (3)  a qualification  of 
the  types  of  error  present.  The  third  objective  is  a quantification  and  summary 
of  the  probable  errors  and/or  error  patterns  present  in  the  data  files. 
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A number  of  researchers  have  stressed  the  potential  for  performing  two 
fundamentally  different  forms  of  quantitative  data  analysis.  These  methods  are 
(1)  the  exploratory  data  analysis  techniques  of  Tukey;  and  (2)  classical 
statistical  techniques. 

Exploratory  data  analysis  is  concerned  with  isolating  patterns  and  features 
of  the  data  and  with  revealing  them  clearly  to  the  analyst.  Ideally, 
exploratory  data  analysis  provides  the  first  contact  with  the  data,  preceding 
any  firm  choice  of  model  for  either  deterministic  or  stochastic  components. 
Exploratory  data  analysis  is  particularly  useful  in  uncovering  unexpected 
departures  from  familiar  models  or  the  assumptions  upon  which  these  models  rest. 
An  important  element  of  the  exploratory  approach  is  flexibility,  both  in 
tailoring  the  analysis  to  the  structure  of  the  data  and  in  responding  to 
patterns  that  successive  steps  of  the  analysis  uncover. 

Specific  techniques  in  exploratory  data  analysis  are  based  on  four  major 
themes:  (1)  resistance;  (2)  residuals;  (3)  re-expression;  and  (4)  display. 

Resistance  is  a matter  of  insensitivity  to  "misbehavior"  in  data.  More 
formally,  an  analysis  or  summary  is  resistant  if  an  arbitrary  change  in  any 
small  part  of  the  data  produces  only  a small  change  in  the  analysis  or  summary. 
For  example,  in  summarizing  the  central  tendency  of  a sample,  the  median  (the 
5Uth  percentile)  is  highly  resistant.  By  contrast,  the  mean  is  highly 
non-resistant.  Consequently,  a number  of  exploratory  techniques  for  structured 
forms  of  data  provide  resistance  because  they  are  based  on  the  median. 

Residuals  are  what  remains  of  the  data  after  a summary  or  fitted  model  has 
been  subtracted  out.  The  attitude  of  exploratory  data  analysis  is  that  an 
analysis  of  a set  of  data  is  not  complete  without  a careful  examination  of  the 
residuals.  This  emphasis  reflects  the  tendency  of  resistant  analyses  to  provide 
a clear  separation  between  dominant  behavior  and  unusual  behavior  in  the  data. 
When  the  bulk  of  the  data  follows  a consistent  pattern,  that  pattern  determines 
a resistant  fit.  The  residuals  then  contain  any  drastic  departures  from  the 
pattern,  as  well  as  the  customary  chance  fluctuations.  Unusual  residuals 
suggest  a need  to  check  on  the  circumstances  surrounding  these  observations. 

Re-expression  involves  the  question  of  what  scale  would  help  to  simplify 
the  analysis  of  the  data.  Exploratory  data  analysis  emphasizes  the  benefits  of 
considering,  at  an  early  stage,  whether  the  scale  in  which  the  data  are 
originally  expressed  is  satisfactory.  The  re-expressions  most  often  used  in 
exploratory  data  analysis  come  from  the  family  of  functions  known  as  power 
transformations  together  with  the  logarithmic  transformation. 

Displays  meet  the  analyst's  need  to  see  behavior  (of  the  data,  of  fits,  of 
diagnostic  measures,  and  of  residuals)  and  thus  to  grasp  the  familiar 
regularities . 

Graphical  analysis  packages  such  as  DATAPLOT  incorporate  all  four  themes  of 
exploratory  data  analysis.  DATAPLOT  is  a FORTRAN-based , interactive  high-level 


31 


language  for  data  analysis  and  graphics.  There  are  six  reasons  why  packages 
like  DATAPLOT  are  recommended  for  use  in  a data  audit.  First,  the  use  of 
graphics  takes  full  advantage  of  human  pattern  recognition  capabilities. 

Second,  graphical  techniques  make  use  of  a minimal  number  of  assumptions,  which 
has  as  an  important  consequence  that  the  conclusions  are  less  likely  to  be 
approach  dependent.  Third,  from  a communications  point  of  view,  graphics  are 
generally  a much  more  understandable  and  efficient  way  of  conveying  information 
to  others  than  is  a set  of  summary  statistics.  Fourth,  graphics  allow  the 
analyst  to  see  and  use  all  of  the  data.  In  forming  a summary  statistic,  this  is 
generally  not  true,  since  information  latent  in  the  entire  data  set  is  mapped 
into  a single  number.  In  general,  the  statistic  is  only  sensitive  to  one 
particular  analysis  aspect  of  the  data.  Fifth,  graphics  permit  the  analyst  to 
check  many  different  aspects  of  the  data  simultaneously.  Information  will 
therefore  be  relayed  not  only  about  what  is  being  investigated,  but  also  about 
unsuspected  anomalies  in  the  data.  Finally,  graphics  complement  classical 
statistical  analysis  techniques.  This  point  will  become  more  clear  as  specific 
techniques  are  described.  Three  types  of  exploratory  data  analysis  techniques 
which  can  be  performed  with  DATAPLOT  are  related  to  measures  of:  (1)  central 

tendency;  (2)  distributional  form;  and  (3)  relational  properties. ^ 

The  use  of  classical  statistical  analysis  in  a data  audit  serves  to  focus 
on  the  same  broad  areas  as  exploratory  techniques,  namely,  central  tendency, 
distributional  form,  and  relational  properties.  Measures  of  central  tendency 
usually  reported  in  statistical  analysis  packages  include  the  mean,  the  median 
and  the  mode  of  the  sample.  Measures  of  dispersion  include  the  variance  and  the 
standard  deviation  of  the  sample.  Measures  of  distributional  form  include  such 
statistics  as  skewness  and  kurtosis  based  on  the  moments  of  the  distribution 
function  as  well  as  histograms  and  goodness-of -f it  tests.  Classical  techniques 
for  analyzing  relational  properties  are  correlation  and  multiple  regression. 
Box-Jenkins  techniques  are  also  of  great  value  in  promoting  a more  complete 
understanding  of  data  having  a time  dimension. 


^A  detailed  description  of  a wide  variety  of  statistical  techniques  for 
performing  a data  audit  is  provided  in  Appendix  B. 
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5.  SENSITIVITY  ANALYSIS 


Sensitivity  analysis  deals  essentially  with  the  response  of  a model  to 
perturbations  (changes)  among  its  state  and  data  variables.  All  of  the  methods 
of  sensitivity  analysis  address  this  underlying  relationship.  The  techniques, 
however,  differ  in  the  way  they  analyze  perturbations,  depending  on  the  nature 
of  the  perturbation  and  its  manifestation  in  the  model  formulation  and  design. 

Conventional  wisdom  dictates  that  models  may  only  be  validated  within  given 
constraints  for  a specific  application.  From  a different  perspective,  however, 
it  may  be  possible  to  validate  a model  in  a universal  sense  if  knowledge  of  the 
subject  is  known  beyond  scientific  doubt  or  is  automatically  acceptable  as  a 
representation  of  a process.  The  techniques  used  in  sensitivity  analyses  are 
themselves  models  of  the  distributional  behavior  of  error,  and  are,  therefore, 
subject  to  the  same  scrutiny  afforded  other  models.  On  the  other  hand,  the 
collective  evidence  of  information  gained  from  the  methodology  provides  a 
powerful  component  to  model  evaluation  procedures. 

5.1  A NOTE  ON  METHODOLOGY 


Sensitivity  analysis  is  usually  essential  in  developing  information 
necessary  for  a comprehensive  assessment  of  model  performance.  It  serves  not 
only  in  the  generation  of  information,  but  contributes  to  understanding  the 
model’s  competence  and  its  range  of  application  to  policy  issues. 

Since  sensitivity  analysis  deals,  in  a broad  sense,  with  model  response  to 
some  condition  or  perturbation,  an  analysis  of  the  response  depends  on:  (1)  the 
type  of  model  under  evaluation;  (2)  the  nature  of  the  condition  or  perturbation 
in  effect;  and  (3)  the  way  the  response  is  measured  and  described.  These  three 
factors  form  the  basis  for  a wide  variety  of  analytical  techniques  which 
collectively  constitute  the  methodology  of  sensitivity  analysis. 


^For  an  in-depth  review  of  the  theoretical  and  empirical  considerations  in  the 
literature  on  sensitivity  analysis,  the  interested  reader  is  referred  to: 
Hendrickson,  R.G.  A Survey  of  Sensitivity  Analysis  Methodology.  Gaithersburg, 
MD:  National  Bureau  of  Standards,  NBSIR  84-2814,  1984. 

O 

^Two  advanced  topics  in  the  theory  of  sensitivity  analysis  which  have  important 
applications  in  the  evaluation  of  policy-oriented  models  are  summarized  in: 
Harris,  C .M . Issues  in  Sensitivity  and  Statistical  Analysis  of  Large-Scale, 
Computer-Based  Models.  Gaithersburg,  MD:  National  Bureau  of  Standards, 

NBS  GCR  84-466",  1984. 
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In  order  to  evaluate  a model  it  may  be  necessary  to  develop  information 
on  model  performance  in  one  or  more  of  the  following  problem  areas  from  which 
error  or  uncertainty  may  originate:  (I)  errors  associated  with  the 

mathematical  formulation  of  the  subject;  (2)  importance  analysis,  which  is  an 
assessment  of  weak  or  non-contributing  data,  representations,  or  levels  of 
aggregation  in  the  design  of  the  model;  (3)  statistical  properties  and  bounds 
of  the  model  response  under  stress  or  test  scenarios;  (4)  stability,  the 
model's  behavior  to  certain  rare  or  unexpected  conditions;  and  (5)  robustness, 
the  performance  of  the  model  when  some  or  all  of  the  assumptions  on  which  the 
model  design  was  based  are  ignored.  Each  of  these  areas  of  potential  interest 
embrace  a variety  of  forms  the  problem  may  take.  Consequently,  the  evaluation 
procedures  which  are  invoked  to  develop  performance  information  in  these  five 
areas  are  themselves  generally  grouped  along  appropriate  lines  of  formal 
mathematical  and  statistical  analysis. 

The  subject  of  error  analysis  covers  a wide  variety  of  types  of  errors 
which  may  be  encountered  during  the  evaluation  of  a model  or  in  a database 
audit.  The  variety  includes  numerical  errors  in  data,  significance  of  values  in 
data  sets,  functional  error  analysis  traditionally  measured  by  means  and 
variances  of  functions  and  represented  by  sensitivity  coefficients  (partial 
derivatives),  propagation  of  errors  through  estimated  computations  in 
subroutines  or  algorithms,  and  sensitivities  as  measured  by  Lagrange 
multipliers . 

The  objective  of  importance  analysis  is  to  identify  those  variables,  or 
subsections  of  a model,  which  do  not  appreciably  affect  the  application  of  the 
results  of  the  model.  The  determination  of  the  relative  weakness  of  variables, 
data,  or  model  components  is  generally  a subjective  matter  to  be  decided  by  the 
evaluator.  Identifying  these  non-contributing  elements,  however,  may  lead  to 
model  simplification,  modification  of  databases,  or  to  the  elimination  of 
unnecessary  complexity  in  the  functional  representation,  of  activities. 

Statistical  analyses  are  the  most  widely  used  methodology  of  sensitivity 
analysis.  They  include  the  calculation  of  mean,  variance,  confidence  or 
tolerance  limits,  regression  and  polynomial  fitting,  hypothesis  testing, 
goodness-of-f i t , covariance,  estimates  of  statistical  parameters,  and 
distribution-free  techniques.  The  information  about  model  performance  gained 
from  statistical  computations  may  be  used  to  analyze  non-s ta tionarity , 
distributional  properties,  variation  of  output,  and  other  measures  of 
sensitivity  of  a stochastic  nature. 

Model  stability  encompasses  three  different  manifestations  of  response: 

(1)  the  relation  of  domain  to  range;  (2)  transition  points  in  model  behavior  as 
a function  of  parametric  trajectories;  and  (3)  the  existence  of  cusps,  or 
catastrophes,  within  the  model  design.  If  the  model  is  viewed  as  a mapping  of 
input  data  (domain)  into  an  output  set  (range),  then  the  evaluator  is  interested 
in  any  anomalies  in  this  operation  which  introduce  abnormal  results  or  mappings 
which  imply  an  increase  in  error  or  decrease  in  the  confidence  of  the  output. 
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Transition  points  are  defined  as  those  points  at  which  the  expected  character  of 
the  model  behavior  undergoes  a significnt  alteration.  They  indicate  to  the 
evaluator  that  the  output  has  taken  a new  direction,  but  are  regarded  to  be  less 
critical  than  the  cusp  or  catastrophic  transition.  The  transition  which  is  a 
catastrophe  is  one  in  which  a critical  state  variable  of  the  model  has  suddenly 
undergone  an  abrupt  change  of  value.  It  usually  represents  either  an  adjustment 
of  the  model  to  stress  or  it  is  a rapid  minimizing  response  to  unstable 
conditions.  The  points  of  transition  are  directly  related  to  values  of 
parameters  which  pursue  a path  in  control  space  (parameter  trajectory).  The 
path,  if  it  moves  into  an  unstable  region  may  produce  either  smooth  transition 
of  a state  variable  or  it  may  trigger  a cusp  or  catastrophe  response,  which  is  a 
discontinuous  jump  in  the  state  variable.  Parameter  tracking  becomes, 
therefore,  an  important  activity  in  the  study  of  model  stability. 


Robustness  in  statistics  is  the  attribute  associated  with  a method  which  is 
successful  although  the  underlying  assumptions  for  its  application  are  violated. 
In  a similar  way,  a model  may  be  said  to  be  robust  if  it  can  be  used  for 
scenarios  it  was  not  designed  to  address,  or  that  it  may  be  used  successfully 
when  its  use  violates  underlying  assumptions  inherent  in  its  design  or 
formulation.  Errors  which  arise  in  this  context  should  be  studied  directly  in 
terms  of  the  assumptions  themselves,  and  also  in  terms  of  the  model  behavior  to 
the  direct  or  indirect  impact  of  these  assumptions  on  the  formulation  of  the 
subject . 


If  a model  displays  robustness  and  can  therefore  be  applied  to  policy 
problems  beyond  those  for  which  it  was  originally  designed,  it  is  desirable  to 
determine,  as  part  of  the  evaluation,  its  range  of  application,  and  to  use  this 
information  to  establish  and  understand  the  model’s  relationship  and  power  to 
similar  models  of  its  own  class,  and  to  identify  the  scope  of  its  multiple 
capabilities  with  respect  to  models  assigned  to  other  classes  of  performance. 


In  addition  to  the  numerical  accuracy  and  significance  of  input  data  it  is 
necessary  to  recognize  two  additional  sources  of  error:  system  noise  and  the 

error  of  observation.  If  the  data  are  not  a time  series  then  the  analytical 
techniques  associated  with  the  classical  theory  of  errors  are  adequate  tools  of 
evalua t i 
examined 
the  data 
regressi 
f ormulat 
non-s tat 


on.  If,  however,  the  data  set  is  a time  series,  then  the  errors  are 
according  to  the  model  of  the  series  and  the  intrinsic  properties  of 
. The  range  of  models  for  time-series  studies  is  quite  large: 
on  at  the  lowest  level,  through  polynomial  and  autoregressive 
ions  to  autoregressive,  integrated,  moving  average  (ARMIA)  models  for 
ionary  time  series. 
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5.2.  THE  METHOD  OF  MODEL  SAMPLING 


Recent  work  by  McKay,  Conover  and  Beckman^  and  by  Harris^ >3 >4  has  produced 
techniques  which  permit  the  effects  of  uncertainty  to  be  rigorously  analyzed. 
Their  approach  makes  use  of  the  method  of  model  sampling.  The  method  of  model 
sampling  is  a procedure  for  sampling  from  a stochastic  process  to  determine, 
through  multiple  trials,  the  nature  and  effects  of  a probability  distribution 
that  would  be  difficult  or  impossible  to  determine  by  standard  statistical 
means.  The  method  of  model  sampling,  or  distribution  sampling  as  it  is  also 
called,  has  a long  history  of  use  by  statisticians  to  derive  distributions 
empirically  that  are  difficult  or  impossible  to  derive  by  other  means. 

Although  the  method  of  model  sampling  is  described  as  an  analytical 
technique  for  performing  a sensitivity  analysis,  it  is  also  one  of  the  most 
appropriate  tools  for  generating  answers  to  "what  if"  type  questions  for  policy 
analyses . 

The  method  of  model  sampling  is  ideally  suited  for  carrying  out  a 
structured  sensitivity  analysis.  In  this  case,  information  on  the  probability 
distribution  for  each  variable  of  interest,  , F1(x^),  is  available  from  the 
data  audit.  Uncertainty  about  the  values  of  the  x^  can  then  be  handled  by 
treating  them  as  random  variables.  In  turn,  this  information  may  be  used  to 
make  inferences  about  the  probability  distribution  of  the  output.  The  mechanics 
of  the  method  of  model  sampling  involves  a "numerical"  experiment  which 
characterizes  the  distribution  of  the  model's  output.  In  essence,  the 
experiment  searches  for  an  unknown  transformation,  h(X) , of  the  vector,  X,  each 
of  whose  components  has  known  probability  distributions,  F1(x][).  Clearly, 
several  sets  of  values  of  X,  say  X^  , ...,  X^,  must  be  selected  as  successive 
sets  of  inputs  in  order  to  determine  the  probability  distribution  of  the 
output . 


^-McKay,  M.D.,  W.  J.  Conover  and  R.  J.  Beckman,  "A  Comparison  of  Three  Methods 
Tor  Selecting  Values  of  Input  Variables  in  the  Analysis  of  Output  from  a 
Computer  Code,"  Technometrics , Vol . 21,  No.  2,  1979,  pp.  239-245. 

■^Harris,  C.  M.  "An  Assessment  of  Climatological  Uncertainties  Using  Monte-Carlo 
Analysis,"  in  Selected  Assessment  Strategies  Applied  to  Short-Term  Energy 
Models . Gaithersburg,  MD:  National  Bureau  of  Standards,  NBSIR  83-2672,  pp . 

30-54,  1983. 

^Harris,  C.  M.  Issues  in  Sensitivity  and  Statistical  Analysis  of  Large-Scale, 
Computer-Based  Models.  Gaithersburg,  MD:  National  Bureau  of  Standards,  NBS 

GCR  84-466,  1984. 

^Harris,  C.  M.  Computer  Generation  of  Latin  Hypercube  Sampling  Plans. 
Gaithersburg,  MD:  National  Bureau  of  Standards,  NBS  GCR  84-476,  1984. 
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There  are  a variety  of  methods  available  for  selecting  the  values  of  the 
vectors.  The  three  most  popular  methods  are:  (1)  random  sampling;  (2) 

stratified  sampling;  and  (3)  Latin  hypercube  sampling.  Random  sampling  has 
intuitive  appeal  in  that  an  entire  body  of  statistical  literature  may  be  used  to 
make  inferences  about  the  distribution  of  the  output.  In  a numerical  experiment 
based  on  the  method  of  random  sampling,  the  values  X]_ , ...,  are  a random 
sample  from  the  individual  distributions,  F1(x-j_).  Although  this  method  produces 
unbiased  estimators  of  the  parameters  of  the  distribution,  it  does  not  guarantee 
that  the  observations  are  reasonably  spaced  over  the  full  range  of  the 
probability  distribution.  This  problem  may  be  quite  serious  if  low-probability 
occurrences  (i.e.,  those  in  the  tails  of  the  distribution)  are  of  importance.  In 
addition,  there  is  no  guarantee  that  values  are  not  duplicated,  which  could 
waste  sampling  resources. 

Some  of  the  difficulties  with  the  method  of  random  sampling  just  mentioned 
have  been  used  to  justify  the  use  of  another  method,  stratified  sampling.  In 
this  method,  all  areas  of  the  sample  space  S of  X are  represented  by  input 
values.  Let  the  sample  space  S of  X be  partitioned  into  I disjoint  strata,  S^, 
of  size 

Pi  = Pr(X  e Si) 

where 

I 

1 Pi  “ 1 
i=l 


Let  X|j , j=l,  ...ni,  be  a random  sample  from  stratum  Si,  where 

l H = N 

i=l 

As  was  the  case  when  the  method  of  random  sampling  was  used,  stratified  sampling 
produces  an  unbiased  estimator  of  the  mean  of  the  output  distribution.  However, 
the  variance  of  the  output  is  less  when  stratified  sampling  is  used  than  when 
random  sampling  is  used.  If  the  probability  sizes,  Pi , of  the  strata  and  the 
sample  sizes,  ni , are  chosen  so  that 

nt  = PiN 

then  a proportional  allocation  is  achieved.  Tocher^  has  shown  that  if 
stratified  sampling  with  proportional  allocation  is  used,  then  the  variance  of 
the  output  distribution  is  reduced  further.  This  implies  that  any  stratified 
plan  which  employs  subsampling,  ni  > 1 , can  be  improved  by  further 
stratification.  This  leads  to  a special  case,  proportional  allocation  with  one 
sample  per  stratum.  As  we  shall  see  shortly,  there  is  a relationship  between 
this  case  and  the  third  method,  Latin  hypercube  sampling. 

•^■Tocher , D.  The  Art  of  Simulation.  Princeton,  NJ : D.  Van  Nostrand,  1963,  pp. 

106-107 . 
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In  stratified  sampling  the  range  space  S of  X can  be  arbitrarily- 
partitioned  to  form  strata.  In  Latin  hypercube  sampling  the  partitions  are 
constructed  in  a specific  manner  using  partitions  of  the  ranges  of  each 
component  of  X.  Consider  the  case  where  k components  of  X are  of  interest.  For 
simplicity,  we  may  assume  they  are  the  first  k components.  Let  the  ranges  of 
each  of  the  k components  of  X be  partitioned  into  N intervals  of  probability 
size  1/N.  The  Cartesian  product  of  these  intervals  partitions  S into  cells 

each  of  probability  N-^.  Each  cell  can  be  labeled  by  a set  of  k cell 
coordinates 

Mi  = ( m± j 1 s •••»  raijk) 

where  m^j^  is  the  interval  number  of  component  x^  represented  in  cell  i.  A 
Latin  hypercube  sample  of  size  N is  obtained  from  a random  selection  of  N of  the 
cells  M ]_ , M^,  with  the  condition  that  for  each  jk  the  set { mi i=l  ,N  is  a 

permutation  of  the  integers  1,  N.  The  columns  of  permutations,  jk,  of  the 

integers  1,  . ..,  N,  must  also  be  created  in  such  a way  that  there  is  no 
duplication  in  any  row.  One  random  observation  is  made  in  each  cell.  The 
method  so  obtained  can  therefore  be  considered  a k-dimensional  extension  of 
Latin  square  sampling. 

McKay,  et  al , were  able  to  show  that  Latin  hypercube  sampling  schemes 
provide  the  most  efficient  means  for  empirically  estimating  the  distribution 
function  of  the  output  as  well  as  the  parameters  of  the  distribution  for  a 
limited  number  of  trials.  One  additional  advantage  of  Latin  hypercube  sampling 
results  when  the  model’s  output  is  dominated  by  only  a few  components  of  X. 

This  is  because  the  Latin  hypercube  sampling  method  ensures  that  each  of  those 
components  is  represented  in  a fully  stratified  manner,  no  matter  which 
components  might  turn  out  to  be  important. 

Whereas  the  construction  of  nonrandom  Latin  hypercube  sampling  schemes  is 
not  a hard  problem,  the  construction  of  random  Latin  hypercube  sampling  schemes 
is  a nontrivial  exercise.  Recent  work  by  Harris  has  resulted  in  a procedure  for 
generating  large-scale  random  Latin  hypercube  sampling  schemes  via  a computer 
code.  This  code  uses  as  arguments  the  values  of  N and  k.  For  example,  if  the 
distribution  of  each  of  four  random  variables  was  divided  into  ten  equally 
probable  increments  (i.e.,  N=10,  k=4),  then  the  procedure  would  generate  a 
sampling  scheme  such  as  is  shown  in  Table  5.1.  For  this  simple  case,  ten 
simulations  would  be  performed  in  the  "numerical"  experiment.  The  probability 
distribution  for  each  variable  F^-(xj_),  F^(x2),  F^(x3)  and  F^(x4)  is  divided  into 
0.10  increments.  Since  N is  even,  the  median  of  each  increment  is  used  (i.e., 
the  5th,  15th,  ...,  95th  percentiles)  for  the  value  of  the  random  variable  x-j_. 
Through  reference  to  Table  5.1  we  see  that  the  first  simulation  would  have 
variable  xy  set  to  the  65th  percentile  of  its  distribution,  X2  to  the  15th 
percentile,  X3  to  the  45th  percentile,  and  X4  to  the  55th  percentile. 
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Table  5.1  A Typical  Randomly  Generated  Latin  Hypercube  Sampling  Scheme 


Simulation 

Variable 

xi 

x3 

x4 

1 

7 

2 

5 

6 

2 

9 

6 

7 

3 

3 

4 

10 

3 

9 

4 

8 

3 

9 

1 

5 

1 

4 

8 

5 

6 

3 

5 

10 

4 

7 

10 

7 

2 

8 

8 

6 

8 

1 

7 

9 

2 

9 

4 

10 

10 

5 

1 

6 

2 
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PART  II 


SELECTED  RESULTS  FROM  AN  EVALUATION 
OF  THE  DYNAMIC  GENERAL  EQUILIBRIUM 
MODEL  (DGEM ) 


40 


ESTABLISHING  USER  REQUIREMENTS  FOR  A MOBILIZATION  APPLICATION! 


6 . 


The  Dynamic  General  Equilibrium  Model  (DGEM)  is  an  annual  model  for 
analyzing  the  structure  and  growth  of  the  U.S.  economy.  DGEM  incorporates  a 
methodology  for  contingency  planning  so  that  quantitative  analyses  of  the  impact 
of  economic  policies  and  disruptions  of  the  U.S.  economy  may  be  conducted.  DGEM 
provides  a detailed  analysis  of  supply  and  demand  factors  for  simulations  up 
through  the  year  2000,  as  well  as  a framework  for  determining  sector-specific 
demands  and  supplies  and  relating  these  developments  to  other  sectors  of  the 
economy.  DGEM  was  developed  by  Dale  W.  Jorgenson  and  Associates;  it  is  an 
expanded  version  of  the  Long  Term  Interindustry  Transactions  Model^  also  known 
as  the  Hudson-Jorgenson  Model. ^ >4  The  model  was  designed  to  deal  with  three 
major  types  of  emergency  situations:  (1)  energy-economic  interactions 

associated  with  supply  interruptions  or  strategic  policies;  (2)  the  economic 
impacts  of  a loss  of  resources  due  to  an  enemy  attack  on  the  United  States  with 
particular  emphasis  on  which  policies  could  stimulate  recovery;  and  (3)  the 
economic  impacts  of  demand  surges  and  resource  constraints  typical  during  a 
period  of  military  mobilization.  The  focus  of  the  material  presented  in  Part  II 
of  this  report  is  on  a military  mobilization  application. 

A military  mobilization  is  a complex  series  of  events  which  requires 
careful  specification  and  in  general,  a variety  of  models  and  databases. ^ One  of 
the  key  components  of  any  such  analysis  is  a macroeconomic  model.  The  purpose 
of  such  models  in  a military  mobilization  is  to  specify  how  the  transition  from 
a peace-time  to  a war-time  economy  takes  place.  Once  this  information  is 
available  to  the  analyst,  it  becomes  possible  to  design  and  evaluate  alternative 
policies  which  should,  at  least  in  theory,  stimulate  or  regulate  certain  sectors 
of  the  economy  which  could  otherwise  produce  shortfalls  or  bottlenecks. 


The  material  presented  in  this  chapter  has  benefited  substantially  from 
information  provided  by:  Prof.  Ernst  R.  Berndt,  Massachusetts  Institute  of 

Technology;  Mr.  David  K.  Henry,  U.  S.  Department  of  Commerce;  and  Messrs.  0. 
Cleveland  Laird,  E.  Laurence  Salkin,  and  Robert  R.  Wilson,  Federal  Emergency 
Management  Agency. 

O 

E.A.  Hudson  and  D.W.  Jorgenson,  The  Long  Term  Interindustry  Transactions 
Model:  A Simulation  Model  for  Energy  and  Economic  Analysis,  Washington,  DC: 

Federal  Preparedness  Agency,  1979. 

^ E.A.  Hudson  and  D.W.  Jorgenson,  "U.S.  Energy  Policy  and  Economic  Growth, 
1975-2000,”  The  Bell  Journal  of  Economics  and  Management  Science,  Vol.  5, 

No.  2,  Autumn  1974,  pp . 461-514. 

^ E.A.  Hudson  and  D.W.  Jorgenson,  "Assessment  and  Selection  of  Models  for  Energy 
and  Economic  Analysis,"  in  Validation  and  Assessment  Issues  of  Energy  Models, 
S.I.  Gass  (Ed.),  Gaithersburg,  MD:  National  Bureau  of  Standards,  NBS 

Special  Publication  569,  1980,  pp.  431-444. 

5 A multi-model  approach  to  military  mobilization  is  outlined  in  Appendix  D. 
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Due  to  the  importance  of  macroeconomic  models  in  a mobilization,  it  is 
useful  to  establish  a set  of  requirements  which  the  model  should  possess.  Four 
general  requirements  are  presented  in  the  text  which  follows.  In  Chapters  7 
through  9,  these  requirements  are  expanded  to  incorporate  the  type  of 
information  described  in  Chapters  2 through  5.  Chapter  10  concludes  with  an 
assessment  of  how  well  DGEM  measures  up  to  these  requirements. 

Requirement  1 

The  model  must  have  documentation  sufficient  to  enable  an  analyst  to: 

(a)  set  it  up  on  the  host  system; 

(b)  execute  a base-case  simulation; 

(c)  interpret  the  results  of  the  base-case  simulation;  and 

(d)  create,  run,  and  interpret  user-specified  simulations. 

Requirement  2 

The  structure  of  the  model  should  be  clearly  documented.  This  requirement 
includes  information  on  the  model’s: 

(a)  policy  context; 

(b)  objectives; 

(c)  exogenous  and  endogenous  variables; 

(d)  equation  structure;  and 

(e)  database(s). 

To  the  extent  feasible,  the  user  should  be  assured  that  the  forecast  values 
of  the  base-case  simulation  have  been  compared  to  historically  achieved 
values.  Any  major  discrepancies  should  be  documented  and,  if  possible, 
explained. 

Requirement  3 

The  model  must  be  cross  referenced  to  the  National  Income  and  Product 
Accounts  (NIPA) . This  minimum  requirement  provides  a set  of  control  totals, 
which  may  be  used  to  disaggregate  the  model’s  output  to  address  sector  specific 
issues.  Furthermore,  it  is  desirable  for  the  model  to  provide  information  on 
those  endogenous  and  exogenous  variables  likely  to  be  of  interest  to  decision 
makers.  This  information  may  be  useful  either  in  gaining  insights  regarding  the 
"path"  of  the  economy  to  a war-time  footing  or  in  preparing  customized  reports. 

Requirement  4 

The  model  must  incorporate  the  following  economic-technical  attributes: 

(a)  the  increasing  importance  of  international  trade  over  the  past 
decade ; 

(b)  the  business  cycle  concept; 

(c)  the  changing  composition  of  the  Gross  National  Product  (GNP); 

(d)  the  concepts  of  investment,  capital  services,  depreciation,  and 
emergency  capacity; 
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(e)  wage  and  price  variables; 

(f)  both  supply  (e.g.,  capital  and  labor  services  available)  and 
demand  (e.g.,  military  requirements)  concepts; 

(g)  an  explicit  treatment  of  both  fiscal  and  monetary  policy; 

(h)  dynamic  characteristics  whereby  production  and  consumption 
decisions  in  one  period  affect  the  economy  in  future  periods;  and 

(i)  bridges  to  or  from  key  variables  (e.g.,  from  the  NIPA  components 
to  specific  sectors  as  defined  by  the  Commerce  Department's 
4-digit  Standard  Industrial  Classification  (SIC)). 

Items  (a)  through  (i)  are  at  the  heart  of  most  mobilization  modeling 
problems.  Consequently,  it  is  essential  that  the  model  provide  a means  through 
which  the  user  can  address  each  item  both  individually  and  in  combination.  Only 
in  this  manner  can  the  importance  of  individual  items  to  the  overall  problem  be 
measured.  For  example,  a model  which  does  not  incorporate  business  cycles  may 
be  unable  to  measure  the  impact  on  wages  and  prices  due  to  a mobilization  which 
•begins  in  a recession  versus  one  which  begins  during  a period  of  full 
employment.  Similarly,  defense  expenditures  are  going  to  exert  a different 
impact  on  certain  sectors  of  the  national  economy  than  are  other  government 
expenditures.  Finally,  the  model  should  provide  sufficient  sectoral  detail  to 
identify  areas  where  in-depth  studies  may  be  worthwhile. 
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7. 


THE  DGEM  ANALYSIS  FRAMEWORK 


This  chapter  addresses  the  issues  discussed  in  Chapters  2 and  3. 

Background  information  is  presented  first;  this  information  serves  to  define  the 
motivation  for  the  model  and  its  theoretical  underpinnings.  The  focus  is  then 
shifted  to  the  DGEM  software,  including:  (1)  a review  of  the  DGEM  source  code; 

(2)  the  implementation  of  the  model  on  the  NBS  computer  system;  and  (3) 
enhancements  which  were  made  to  the  basic  code.  The  section  concludes  with  a 
report  on  NBS*  experiences,  as  well  as  those  of  several  other  Federal  agencies, 
who  used  the  model  to  reproduce  the  base-case  simulation  and  then  performed 
certain  mobiliza tion=oriented  studies. 


7.1  THE  DGEM  DOCUMENTATION 


The  material  which  follows  is  based  on  a thorough  review  of  the  DGEM 
documentation  reports.  In  some  cases,  the  material  presented  incorporates 
information  provided  to  the  NBS  staff  by  the  model  developers.  Most  of  this 
information  was  prompted  by  the  need  for  NBS  to  bring  the  DGEM  code  into 
compliance  with  the  ANSI  X3e 9-1978  standard  for  FORTRAN  77.  Although  this 
information  delves  deeper  into  the  documentation  than  would  be  required  for  an 
analyst  to  understand  and  use  the  model,  it  helps  to  integrate  material  from 
several  different  documentation  reports  into  an  overall  picture  of  how  the  model 
operates . 


From  a review  of  the  documentation,  it  is  clear  that  much  care  went  into 
the  design,  development  and  testing  of  the  model.  The  documentation  is  quite 
complete  and  should  be  sufficient  for  an  analyst  to  setup  and  run  the  model  on 
the  host  system  and  interpret  its  output. 


The 

consists 


documentation  reports  were  prepared  in  two  stages, 
of  a set  of  reports  on: 


The  first  stage 


(1)  household-*-  and  producer^  behavior; 

(2)  comparisons  of  the  DGEM  predictions  against  realized  values  up 
through  1974; 3 and 

(3)  results  from  a series  of  tests  on  the  convergence  properties  of  the 
model  for  extreme  changes  in  certain  key  variables.^ 


^ The  36  Sector  Model  of  Household  Behavior,  Lexington,  MA:  Data  Resources, 

Inc . , 1979. 

“ The  36  Sector  Model  of  Producer  Behavior,  Lexington,  MA:  Data  Resources, 

Inc.,  1979. 

^ Simulations  of  the  36  DGEM  Over  the  Historical  Period,  Lexington,  MA:  Data 
Resources,  Inc.,  1980. 

^ Test  and  Sensitivity  Simulations  of  the  36  Sector  Dynamic  General  Equilibrium 
M odel , Lexington,  MA:  Data  Resources,  Inc.,  1980. 
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The  second  stage  consists  of  a set  of  reports  giving: 

(1)  explicit  instructions  for  applying  the  model  to  a wide  variety  of 
emergency  situations; 1 

(2)  descriptions  of  key  variables,  comparisons  of  how  the  model  performed 
outside  the  estimation  period,  and  details  of  the  base-case  projection 
through  the  year  2000 and 

(3)  a mathematical  description  of  the  model,  including  key  equations,  and 
the  solution  algorithm. ^ 


The  approach  used  in  DGEM  is  based  on  the  application  of  econometric 
modeling  to  input-output  analysis.  Where  input-output  analysis  assumes  fixed 
input-output  coefficients  at  any  point  in  time,  DGEM  provides  for  flexible 
input-output  coefficients  induced  by  price  variations  in  primary  inputs  which 
are  associated  with  economic  policies  or  anticipated  contingencies.  The 
complete  model  consists  of  an  inter-industry  model  incorporating  the  flexible 
input-output  methodology  and  a macroeconometric  model  that  integrates  demand  and 
supply  conditions  for  consumption,  investment,  capital  and  labor. 


The  macroeconometric  model  divides  economic  activity  into  four  types  of 
goods  and  services:  the  output  of  consumption  and  investment  goods;  and  the 

inputs  of  capital  and  labor.  A production  function  relates  the  output  of 
consumption  and  investment  goods  to  the  inputs  of  capital  and  labor  services, 
for  a given  level  of  technical  efficiency. 


The  inter-industry  model  determines  inter-industry  transactions  for  36 
domestic  sectors  (see  Table  7.1),  the  demand  for  primary  inputs,  the  allocation 
of  GNP  as  final  demand  among  the  sectors,  and  the  total  sector  outputs.  The 
technology  of  each  producing  sector  is  represented  by  a price  possibility 
frontier  that  determines  the  supply  price  of  output  as  a function  of  the  prices 
of  primary  and  intermediate  inputs  and  the  level  of  technical  efficiency. 


Each  of  the  36  sectors  in  the  domestic  economy  is  represented  by  a submodel 
of  producer  behavior.  These  submodels  are  based  on  the  translog  price 
possibility  frontier.  This  frontier  is  a function  relating  the  price  of  output 
charged  by  a sector  to  the  prices  that  the  sector  pays  for  its  inputs.  Output 
price  equals  average  cost,  with  profit,  in  the  form  of  return  to  capital,  being 
included  in  this  cost.  Also,  technical  change  is  included  in  the  price 
frontiers . 


^ R.J.  Goettle  and  E.A.  Hudson,  User’s  Guide  to  the  36  DGEM  Simulation  Model, 
Cambridge,  MA:  Dale  Jorgenson  and  Associates,  1984. 

O 

R.J.  Goettle  and  E.A.  Hudson,  Final  Report  on  the  Dynamic  General  Equilibrium 
M odel , Cambridge,  MA:  Dale  Jorgenson  and  Associates,  1984. 

^ R.J.  Goettle  and  E.A.  Hudson,  36  DGEM : The  Dynamic  General  Equilibrium 
Simulation  Model,  Cambridge,  MA:  Dale  Jorgenson  and  Associates,  1984. 
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The  household  sector  is  also  explicitly  analyzed,  not  only  in  terms  of  its 
demand  for  the  output  of  the  producing  sectors  but  also  in  terms  of  the  supply 
of  labor  and  the  volume  of  saving.  Part  of  the  model  is  organized  within  an 
interindustry  transactions  framework.  This  permits  balance  and  consistency 
between  input  and  output  patterns  to  be  achieved  over  all  intermediate  and  final 
goods  markets.  Another  part  of  the  model  covers  the  supply  of  primary  inputs, 
in  particular  of  capital  and  labor,  the  demand  for  these  inputs  and  the 
adjustment  of  activity  patterns  so  that  the  input  markets  are  in  balance.  The 
model  also  covers  the  growth  of  the  economy  over  time  with  explicit  attention 
given  to  savings  and  investment  mechanisms  and  expansion  of  productive 
capacities  through  increases  in  capital  and  labor  input  and  improvements  in 
technical  efficiencies. 

The  analytical  framework  in  DGEM  incorporates  several  key  factors.  The 
consistency  of  the  framework  ensures  that  the  quantity  and  the  value  of  flows  in 
each  market  in  the  economy  are  simultaneously  in  balance.  Both  price  and 
quantity  aspects  of  economic  activity  are  explicitly  included  in  the  model. 
Behavior  by  producers  and  consumers  is  considered  in  both  price  and  quantity 
terms.  For  producers,  the  formation  of  output  prices  is  considered  as  well  as 
the  selection  of  those  input  patterns  that  are  appropriate  in  the  face  of  these 
prevailing  input  and  output  prices  (i.e.,  the  determination  of  the  input-output 
coefficients  for  each  producer  as  a function  of  technological  information  and  of 
prevailing  prices).  For  the  household  sector,  prices  enter  the  determination  of 
labor  supply,  of  total  consumption  expenditure,  and  of  saving.  In  addition,  the 
quantities  of  household  demands  for  produced  goods  and  services  are  functions  of 
preference  parameters  and  the  prevailing  price. 

The  flow  of  inputs  to  and  outputs  from  production  is  handled  within  an 
interindustry  transaction  framework.  In  this  framework  the  transactions  are 
organized  in  a matrix  with  each  column  representing  inputs  to  an  industry  and 
each  row  representing  sales  or  output  from  an  industry.  Each  row  corresponds  to 
supply  from  a sector;  each  column  represents  purchases  by  a sector.  There  are 
36  producing  sectors.  There  are  three  further  sources  of  supply:  (1)  capital 

services;  (2)  labor  services;  and  (3)  imports.  Also,  there  are  four  more 
purchasing  sectors,  the  final  demand  activities.  These  are:  (1)  personal 

consumption;  (2)  investment;  (3)  government  purchases;  and  (4)  exports. 

To  summarize,  DGEM  is  comprised  of  several  components. 

1.  Submodels  of  producer  behavior,  one  for  each  of  the  36  domestic 
producing  sectors. 

2.  A model  of  consumer  behavior. 

3.  Balance  equations  covering  physical  flows  through  the  interindustry 
system  equating  demand  and  supply  quantities  of  each  good  or  service 
transacted . 

4.  Market  balance  equations  equating  value  of  expenditure  and  receipts 
for  each  good  or  service  transacted. 

5.  Financial  identities  aggregating  value  flows  into  aggregate  income, 
financial  and  economic  accounts. 

6.  Government  and  rest  of  the  world  accounts. 
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Table  7.1  The  DGEM 

Sectoring  Scheme 

Sector  Number 

Sector  Name 

1 

Agriculture,  Forestry  and  Fisheries 

2 

Metal  Mining 

3 

Coal  Mining 

4 

Crude  Petroleum  and  Natural  Gas 

5 

Nonmetallic  Mining  and  Quarrying,  except  Fuel 

6 

Construction 

7 

Food  and  Kindred  Products 

8 

Tobacco  Manufactures 

9 

Textile  Mill  Products 

10 

Apparel  and  Other  Fabricated  Textile  Products 

11 

Lumber  and  Wood  Products 

12 

Furniture  and  Fixtures 

13 

Paper  and  Allied  Products 

14 

Printing,  Publishing  and  Allied  Industries 

15 

Chemicals  and  Allied  Products 

16 

Petroleum  Refining 

17 

Rubber  and  Miscellaneous  Plastic  Products 

18 

Leather  and  Leather  Products 

19 

Stone,  Clay,  and  Glass  Products 

20 

Primary  Metal  Industries 

21 

Fabricated  Metal  Products 

22 

Machinery,  except  Electrical 

23 

Electrical  Machinery 
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Table  7.1  The  DGEM  Sectoring  Scheme  (continued) 


Sector  Number 

Sector  Name 

24 

Motor  Vehicles  and  Motor  Vehicles  Equipment 

25 

Transportation  Equipment  & Ordnance,  except  Motor 

Vehicles 

26 

Instruments 

% 

27 

Miscellaneous  Manufacturing  Industries 

28 

Transportation 

29 

Communications 

30 

Electric  Utilities  (including  Federal,  state,  and 

local ) 

31 

Gas  Utilities 

32 

Trade 

33 

Finance,  Insurance,  and  Real  Estate 

34 

Services  (including  water  and  sanitary  services) 

35 

Government  Enterprises  (excluding  electric  utilities) 

36 

Miscellaneous 
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7.2  SOFTWARE  CONSIDERATIONS 


The  current  version  of  DGEM  is  written  in  FORTRAN  77  and  designed  for 
execution  on  any  mainframe  or  mini  which  supports  the  ANSI  X3. 9-1978  standard 
for  FORTRAN.  During  the  review  of  the  source  code,  which  was  provided  to  NBS  by 
FEMA,  it  became  obvious  that  several  modifications  would  be  required  if  the 
model  were  to  run  on  equipment  other  than  the  UNIVAC  1100  series  of  computers, 
the  original  target  machine  configuration.  The  review  of  the  source  code  did, 
however,  demonstrate  that  the  description  of  the  mathematical  model  given  in  the 
36  DGElT  : The  Dynamic  General  Equilibrium  Simulation  Model  report  is  an  adequate 
description  of  the  operational  model. 

7.2.1  Code  Review  and  Modification 

The  discussion  which  follows  first  focuses  on  a brief  description  of  the 
solution  procedure.  This  discussion  parallels  that  given  in  the  36  DGEM : The 
Dynamic  General  Equilibrium  Simulation  Model  report.  Emphasis  is  then  placed  on 
documenting  the  changes  in  the  DGEM  source  code  resulting  from  its  implementa- 
tion on  selected  mini  and  mainframe  computer  systems.  A one  page  description  of 
each  subroutine,  which  identifies  specific  code  modifications,  is  given  in 
Appendix  F. 

The  basic  approach  of  the  DGEM  model  is  to  generate  a numerical  solution 
for  a set  of  simultaneous  equations.  The  simulation  program  provides  a frame- 
work for  this  solution  and  for  the  handling  of  the  large  amounts  of  data  and 
information  generated.  The  components  of  the  program  are  subroutines  that 
handle  data  input,  subroutines  that  incorporate  the  equations  of  the  model, 
procedures  for  numerical  solution  of  equation  systems,  and  subroutines  for  the 
handling  of  solution  information.  These  are  organized  into  a consistent  frame- 
work by  the  MAIN  program. 

The  logical  sequence  of  the  simulation  is  as  follows.  First,  control  in- 
formation such  as  the  simulation  period  and  the  nature  of  any  alterations  to  the 
system  is  inserted.  Then,  the  database  is  assembled  from  the  data  file.  This 
database  comprises  the  values  of  the  endogenous  and  exogenous  variables  over  the 
simulation  period  and  the  values  of  the  coefficients  or  parameters  of  the  model 
system.-*-  Any  user-specified  changes  in  data  or  assumptions  are  also  introduced. 
Next,  the  equations  of  the  model  are  introduced.  Then,  sequentially,  for  each 
year  of  the  simulation  period,  the  equation  system  is  solved,  given  the  values 
of  the  exogenous  variables  and  the  coefficients.  For  each  year's  solution,  the 
numerical  values  of  the  exogenous  variables,  the  lagged  endogenous  variables, 
and  the  coefficients  are  inserted  into  the  equations.  This  generates  a system 
of  N equations  involving  N variables,  the  N endogenous  variables.  A numerical 
solution  algorithm  is  then  used  to  ascertain  these  solution  values  for  the 
endogenous  variables.  The  values  of  the  endogenous  variables  are  stored  for 
use  in  the  next  year’s  simulation,  as  lagged  endogenous  variables.  Also,  the 


1 The  data  file,  from  which  the  database  for  the  simulation  is  developed, 
contains  information  covering  the  years  1958  through  2000.  The  post  1982 
values  in  the  data  file  are  projections,  based  on  data  available  in  1983, 
which  include:  (1)  trends  in  real  economic  growth  and  inflation;  (2)  the 

business  cycle;  (3)  aggregate  sources  of  growth;  (4)  trends  in  final  demand; 
(5)  energy  use;  and  (6)  industry  developments. 
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solution  values  are  saved  for  display  at  the  conclusion  of  the  simulation.  This 
solution  sequence  is  then  repeated  for  each  year  of  the  simulation  period. 
Finally,  the  solution  information  for  the  entire  simulation  period  is  displayed 
in  the  form  and  for  those  variables  requested  by  the  user. 

The  structure  of  the  DGEM  simulation  program  is  illustrated  in  Figure  6.1. 
Figure  6.1  shows  the  principal  information  flows  in  the  program  and  indicates, 
in  capitals,  the  principal  subroutines  involved  at  each  step.  The  program  first 
collects  control  information  through  subroutines  CHROPT  and  CNTRLR.  These 
introduce  run  control  options,-*-  the  initial  and  final  simulation  years,  whether 
the  simulation  is  static  or  dynamic, ^ the  number  and  specification  of  endogenous 
variable  constraints  to  be  imposed  on  the  model,  the  number  and  specification  of 
data  changes  to  be  introduced,  and  a series  of  print  controls.  Subroutine  DATA 
is  then  called  to  read  the  data  for  the  endogenous  variables,  the  exogenous 
variables,  the  coefficients,  the  values  for  each  constraint  variable,  and  any 
adjustments  in  the  exogenous  variables.  The  values  of  the  lagged  endogenous 
variables  for  this  simulation  year  are  then  assembled  by  subroutine  SAVLAG, 
drawing  on  the  previous  yearfs  data  values  (for  the  first  year)  of  the 
endogenous  variables.  Under  some  output  specifications,  actual  values  of 
user~=selected  variables  will  be  saved  here  so  that  they  can  be  compared,  after 
solution,  with  the  simulated  values  of  these  variables;  subroutine  SAVOUT  is 
concerned  with  this  data  storage.  At  this  point,  all  the  information  is  in 
place  for  the  initiation  of  the  simulation.  As  the  model  equations  are  solved 
numerically,  an  initial  guess  of  the  values  of  the  endogenous  variables  is 
required.  Specifically,  only  N(N>40)  endogenous  variables  are  solved  for 
numerically  in  the  reduced  dimension  system,  so  initial  guesses  on  these  N basis 
variables  are  required. 3 These  guesses  are  selected  by  subroutine  GUESS, 
drawing  on  the  actual  values  of  the  endogenous  variables.  The  solution 
procedure  is  now  invoked;  this  is  handled  by  subroutine  NEWTM , which  begins  at 
statement  label  C.  The  data  (exogenous  variables,  lagged  endogenous  variables, 
and  coefficients)  are  inserted  into  the  model  equations,  contained  in 
subroutines  FCR,  FCRVAL,  SHPRA,  FCRA,  FCRB,  and  SHPRB.  The  equations  are 
evaluated  given  the  current  guess  of  the  values  of  the  endogenous  variables. 


*-  Examples  of  run  control  options  are:  (I)  output  the  National  Income  and 

Product  Accounts  for  each  year;  (J)  print  a table  of  energy  aggregates  for 
each  year;  (M ) create  a new  data  file;  and  (R)  suppress  the  standard  output 
whenever  reports  on  special  variables  are  desired. 

O 

^ In  a dynamic  simulation  the  solution  values  for  one  year  form  the  initial 
conditions  for  the  following  year. 

3 The  value  of  N is  at  least  40.  In  this  case,  the  first  variable  is  IVT,  the 
next  36  variables  correspond  to  the  vector  PFM(i),  the  last  three  variables 
are  PKD,  PLM  , and  RNW,  respectively.  In  cases  where  N is  greater  than  40,  the 
first  40  variables  are  as  defined  above  and  the  basis  includes  one  additional 
variable  for  each  equality  constraint.  The  variable  corresponding  to  each 
equation  is  the  instrument  nominated  by  the  user. 
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Figure  7.1  Flowchart  of  the  DGEM  Computer  Program 
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Figure  7.1  Flowchart  of  the  DGEM  Computer  Progarm  (continued) 
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Figure  7.1  Flowchart  of  the  DGEM  Computer  Program  (continued) 
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This  evaluation  is  partly  a matter  of  recursive  equation  solution  but  also 
involves  the  numerical  solution  of  three  blocks  of  equations  (i.e.,  there  are 
three  separate  numerical  solutions  within  the  outside  numerical  solution  loop). 
One  block  is  solved  for  the  output  prices  using  a Newton’s  method  procedure  in 
NEWTMA  that,  in  turn,  employs  FCRA  and  SHPRA;  the  next  block  involves  solving 
for  the  pattern  of  household  sector  purchases  which  uses  subroutines  NEWTMB, 
FCRB,  and  SHPRB;  and  the  third  block  involves  the  interindustry  transactions 
balances  and  is  solved  by  the  linear  equation  solver  DECSOL.  If  the  equations 
hold  exactly  (i.e.,  the  N residuals  corresponding  to  the  basis  variables  are 
simultaneously  zero)  the  current  endogenous  variable  values  are  solution  values 
and  no  further  processing  is  required  (i.e.,  passes  out  through  statement  label 
E).  If  the  residuals  are  not  zero,  the  outside  loop  Newton's  method  algorithm 
commences  a systematic  adjustment  of  their  values  (i.e,  returns  to  statement 
label  C).  After  each  adjustment,  the  entire  model  is  re-evaluated  and  the 
residuals  computed.  When  the  residuals  are  simultaneously  zero,  model  solution 
has  been  achieved  and  the  adjustment  procedure  is  halted.  The  solved  values  are 
saved,  by  SAVLAG,  to  provide  the  lagged  values  for  the  next  year's  simulation. 
Also,  the  solved  values  are  stored,  for  display  or  analysis  after  the  simulation 
has  been  completed.  Subroutine  SAVOUT  saves  specific  information  for  OUTPUT; 
0UU10D  organizes  and  prints  blocks  of  variables.  The  program  now  moves  to  the 
next  year  and  repeats  the  solution  process.  This  continues  until  the  system  has 
been  solved  for  every  year  of  the  specified  simulation  period.  Finally,  the 
output  information  requested  by  the  user  is  displayed  from  subroutine  OUTPUT. 


The  method  of  evaluating  the  equations  of  DGEM  is  itself  complicated.  This 
solution  is  handled  by  the  Newton's  method  procedure  in  subroutine  NEWTT1  which 
calls  subroutine  FCR  to  evaluate  the  equations.  The  equations  are  set  up  as  a 
sequence  of  equation  blocks  that  can  be  solved  conditionally  on  the  values  of  N 
basis  variables.  The  purpose  of  FCR  is  to  evaluate  N residual  equations 
corresponding  to  these  basis  variables;  NEWTM  adjusts  the  values  of  these 
variables  until  the  residuals  are  simultaneously  zero.  Within  FCR,  the  entire 
system  of  equations  must,  be  evaluated.  FCR  inserts  the  values  of  the  basis 
variables  into  these  variables  by  name  then  calls  subroutine  FCRVAL.  FCRVAL 
calculates  the  primary  input  prices  and  makes  initial  guesses  on  the  output 
prices  before  calling  NEWTMA  to  solve  numerically  for  the  output  prices.  NEWTMA 
calls  subroutine  FCRA  (which,  in  turn,  uses  SHPRA)  to  calculate  input  costs, 
input  shares,  production  costs  and,  from  these,  the  residuals  for  the  output 
prices  for  flexible  share  sectors.  It  also  calculates  average  price  residual 
for  fixed  coefficient  sector  (i.e.,  sector  36).  Once  the  output  prices  have 
been  determined,  FCRVAL  proceeds  to  construct  initial  estimates  of  the 
expenditure  weighted  average  price  facing  households,  PCC.  NEWTMB  is  then  used 
to  numerically  solve  for  this  consumption  price.  The  equations  used  in  this 
solution  are  in  FCRB  and  SHPRB  and  cover  the  determination  of  consumption 
expenditure  patterns.  With  these  consumption  prices  and  consumption  patterns, 
FCRVAL  proceeds  to  calculate  all  production  and  final  demand  prices,  to  find  all 
final  demands,  and  to  compute  the  input-output  coefficients.  Next,  the 
input-output  system  is  solved  numerically  using  DECSOL,  a linear  equation 
solver.  From  the  resulting  sectoral  outputs  the  entire  matrix  of  real 
transactions  is  calculated.  Next,  the  financial,  government  and  foreign  trade 
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variables  are  calculated.  At  this  point,  all  equations  and  variables  in  the 
model  have  been  evaluated  except  for  the  N equations  corresponding  to  the  basis 
variables.  The  program  now  returns  to  FCR  where  these  N equations  are 
evaluated,  yielding  the  residuals  used  by  NEWTM  in  solving  for  the  N basis 
variables . 

Shortly  after  our  initial  review  of  the  DGEM  documentation  and  source  code 
was  completed,  NBS  installed  a new  computer  system.  The  new  system  consists  of 
a CDC  Cyber  205  (622  series)  supercomputer  coupled  with  a CDC  Cyber  180/855 
mainframe  computer.  The  Cyber  855  is  used  as  the  front-end  system  for  the 
supercomputer.  For  a brief  period  during  the  summer  of  1985,  both  the  original 
NBS  Sperry  UNIVAC  1100/82  system  and  the  new  Cyber  205-855  were  operating 
concurrently.  This  enabled  the  NBS  team  to  perform  extensive  testing  and  code 
revisions  simultaneously  on  both  machines.  Side-by-side  comparisons  of  the 
model's  output  were  then  performed  to  catch  errors  introduced  by  the  code 
revisions  or  to  isolate  second-order  effects.  This  approach  was  taken  to  ensure 
that  changes  in  the  model's  logic  were  not  inadvertently  introduced  into  the 
source  code.  Copies  of  the  revised  source  code  and  a sequential  version  of  the 
data  file  were  provided  to  the  developers  for  their  review.  Several 
enhancements  were  subsequently  incorporated  by  the  developers  to  facilitate  the 
use  of  the  Mobilization  TUTOR  described  in  Section  7.2.2.  The  discussion 
which  follows  incorporates  these  enhancements. 

The  revised  code  now  satisfies  the  ANSI  X3. 9-1978  standard  for  FORTRAN  and 
is  operational  on  the  NBS  Cyber  205-855  system  and  the  Center  for  Computing  and 
Applied  Mathematics  VAX/VMS  11-785  minicomputer.  Major  revisions  to  the  DCEM 
source  code  were  of  three  types.  All  three  types  of  revisions  were  due  to  the 
use  of  Sperry  UNIVAC  extensions  to  the  FORTRAN  standard.  The  first  set  of  code 
revisions  is  related  to  the  use  of  INCLUDE  statements  throughout  the  model.  The 
second  set  of  revisions  involved  four  blocks  of  subroutines;  these  blocks  are 
associated  with  the  following  subroutines:  (1)  DECSOL;  (2)  FCRA;  (3)  FCRB;  and 

(4)  GETOUT.  The  third  set  of  revisions  involved  character  string 
manipulations,  most  notably  within  the  original  GETOUT  block  of  subroutines 
referred  to  hereafter  as  the  "parser."  Minor  revisions  to  the  DGEM  source  code 
included:  (1)  file  definition  statements;  (2)  the  initialization  of  certain  key 

variables  within  subroutines;  and  (3)  settings  for  the  variables  used  to  test 
for  convergence.  Brief  descriptions  of  each  type  of  revision  are  given  in 
the  text  which  follows.  Subroutine-by-subroutine  descriptions  of  code  revisions 
are  given  in  Appendix  F. 

The  INCLUDE  statement  is  a Sperry  UNIVAC  extension  to  the  ANSI  X3 . 9-1978 
standard  for  FORTRAN;  it  inserts  an  externally  defined  set  of  FORTRAN  statements 
into  the  program  being  compiled.  The  Sperry  UNIVAC  Procedure  Definition 
Processor  (PDP)  is  used  to  create  a FORTRAN  PROC.  This  is  the  set  of  FORTRAN 
statements  which  can  be  inserted  into  the  source  language  with  an  INCLUDE 
statement.  The  PDP  accepts  the  source  language  statements  defining  FORTRAN 
procedures  and  builds  an  element  in  the  user-defined  program  file.  By  using 
INCLUDE  statements,  these  procedures  may  be  referenced  subsequently  in  a 
compilation  without  redefinition.  This  approach  saves  both  time  in 
recompilation  and  space  in  the  program  listing. 
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A normal  application  for  INCLUDE  procedures  is  one  or  more  INCLUDE  elements 
containing  a set  of  data  declarations  shared  between  different  portions  of  a 
larger  user  program.  In  the  case  of  DGEM , INCLUDE  statements  were  used  to  refer 
to  named  COMMON  blocks  and  the  PARAMETER  block.  This  convention  prompted  a 
replacement  of  the  INCLUDE  statements  with  explicit  COMMON  block  statements  and, 
where  appropriate,  with  the  PARAMETER  block  statements.  In  addition,  it  was 
necessary  to  replace  the  FORTRAN  procedures  with  a BLOCK  DATA  subprogram.  Under 
the  ANSI  X3. 9-1978  standard,  BLOCK  DATA  subprograms  are  used  to  specify 
parameters  and  to  provide  initial  values  for  variables  and  elements  in  named 
COMMON  blocks.  Brief  descriptions  of  all  DGEM  named  COMMON  blocks  are  given  in 
Table  7.2. 


A subprogram  is  a program  unit  that  has  a FUNCTION,  SUBROUTINE,  or 
BLOCK  DATA  statement  as  its  first  statement.  A subprogram  whose  first  statement 
is  a SUBROUTINE  statement  is  called  a subroutine  subprogram. 

Under  the  Sperry  UNIVAC  implementation  of  FORTRAN,  a subroutine  subprogram 
may  be  referred  to  either  as  external  or  internal.  A subroutine  is  external  if 
it  appears  as  the  first  program  unit  in  the  source  input,  or  its  SUBROUTINE 
statement  is  immediately  proceeded  by  an  END  statement.  Otherwise  the 
subroutine  is  internal.  The  Sperry  UNIVAC  implementation  of  FORTRAN  permits  an 
external  subroutine  subprogram  to  contain  several  internal  subroutines  (i.e., 
several  SUBROUTINE  statements).  As  indicated  earlier,  there  were  four  cases  in 
which  an  external  subroutine  subprogram  contained  more  than  one  SUBROUTINE 
statement . ^ 


1 In  the  original  source  code,  subroutine  OUTPUT  contained  two  FUNCTION 
statements.  Subroutine  OUTPUT  was  revised  in  the  same  way  as  subroutines 
DECSOL,  FCRA,  FCRB,  and  GETOUT  (see  the  text  which  follows).  However,  the 
changes  to  subroutine  OUTPUT  were  relatively  straight  forward. 
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Table  7.2  DGEM  COMMON  Blocks 


Name 

Purpose 

B 

CHRSET3 
CNTRL 
COEFF 
DIMSIZ3 
‘ ENDOG 

EXOG 
ICOB 
LAG 
OPTb 
OPTN3 
OTPTNM 
OUTPTA 
OUTPTB 
TARINF 
TIERH 
TIERP 

Variable  names  and  coefficient  names 
Host’s  FORTRAN  character  set 
Control  Parameters 
Coefficients 

Dimensions  for  special  output  variables 
Endogenous  variables 
Exogenous  variables 
Cobb-Douglas  specifications 
Lagged  variables 
Defines  DEBUG 
Defines  DEBUG 
Information  on  special 
variables  for  which 
the  user  wants  reports 
Information  on  user-controlled  variables 
Indices  for  the  household  model 
Indices  for  the  producer  models 

3 New  COMMON  block 
b No  longer  used 
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Under  the  Sperry  UNIVAC  implementation,  entities  that  may  be  referenced 
from  both  external  and  internal  program  units  are  referred  to  as  global. 

Entities  which  may  be  referenced  only  within  a particular  internal  subprogram 
are  referred  to  as  local  (i.e.,  local  to  that  subprogram).  Entities  used  in  an 
external  program  unit  can  not  be  referenced  by  another  external  program  unit 
unless  they  are  in  COMMON  blocks  or  passed  as  arguments.  Variables,  arrays, 
parameter  variables,  and  so  forth,  which  are  declared  or  used  in  an  external 
program  unit  can  be  referenced  by  any  of  its  internal  subprograms. 

Consequently,  the  names  of  certain  entities  need  not  be  used  as  arguments  in 
calling  an  internal  subroutine. 

The  code  revisions  prompted  by  the  Sperry  UNIVAC  extensions  to  the  FORTRAN 
standard  were  of  two  basic  types.  First,  each  subroutine  was  made  external. 

This  was  done  by  ensuring  that  an  END  statement  was  the  last  line  of  each 
subroutine.  Second,  the  calling  sequence  for  each  subroutine  which  was  formerly 
internal  now  includes,  as  explicit  arguments,  the  names  of  entities  which  are 
referenced  within  it  and  which  were  previously  local.  For  example,  the  calling 
sequence  for  subroutine  OFFSET  within  subroutine  GETOUT  was  changed  from 

CALL  OFFSET(IVAR) 
to 

CALL  OFFSET( IVAR , SUB 1 , SUB2 , SUB3 , SUB4 , POINT , INREC , KEYVAR , LOCVAR) 

because  variables  SUB1  through  LOCVAR  were  previously  treated  as  local 
variables . 

The  purpose  of  the  parser  is  to  read  the  run  control  information.  This 
includes  the  specification  of  User/DEBUG  options  (see  Table  7.3),  the  first  and 
last  years  of  the  simulation,  whether  the  simulation  is  static  or  dynamic  and 
the  numbers  of  target  instrument  pairs,  data  changes  and  outputs.  The  names  and 
values  of  the  variables  are  also  read. 

The  need  to  revise  the  parser  was  due  to  the  use  of  word-length  dependent 
software  designed  for  execution  on  Sperry  UNIVAC  equipment.  This  software  was 
very  flexible  in  that  it  enabled  the  analyst  to  operate  on  bit  strings. 
Unfortunately,  the  Sperry  UNIVAC  operates  on  a 36-bit  word.  Thus,  word  bytes 
and  OCTAL  field  codes  which  operate  on  Sperry  UNIVAC  equipment  will  either  not 
work  or  may  produce  misleading  results  on  other  machines.  For  example,  the  FLD 
function  used  in  subroutine  PACK  operates  on  the  Fieldata  code  set  rather  than 
the  ASCII  code  set.  Although  all  Fieldata  characters  have  corresponding 
representations  in  ASCII,  characters  are  four  to  a word  rather  than  six  and  will 
have  different  internal  representations.  A similar  problem  results  from  using 
INTEGER  declarations  to  store  6-character  strings  or  multiples  thereof. 
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Table  7.3  User/DEBUG  Options  for  DGEM 


OPTION 

FUNCTION 

B 

Suppress  all  Newton's  method  output. 

C 

In  subroutine  FCRVAL,  suppress  the  calls  to  the  FCRA  and  FCRB 
subroutines  (i.e.,  compute  only  those  variables  listed  in  FCRVAL). 

D 

Suppress  all  lagged  endogenous  variable  output. 

E 

Call  subroutine  OUTMOD  after  the  first  time  through  FCRVAL. 

F 

List  the  residuals  from  NEWTMA,  the  numerical  solution  for  PO. 

G 

List  the  residuals  and  the  expenditure  share  from  NEWTMB,  the 
numerical  solution  for  PCC. 

H 

Compute  only  the  financial  variables  in  subroutine  FCRVAL,  omitting 
all  other  equations  and  suppressing  the  calls  to  the  FCRA  and  FCRB 
subroutines . 

I 

Output  National  Income  and  Product  Accounts  for  each  year. 

J 

Print  table  of  energy  aggregates  for  each  year. 

K 

List  PO  every  time  subroutine  FCRVAL  is  called. 

L 

List  household  prices  every  time  through  subroutine  FCRVAL. 

M 

Overwrite  the  file  database  with  the  database  created  in  this  run. 
(Use  this  option  with  caution  — ■ the  previous  database  will  be  lost.) 

R 

Suppress  standard  output  whenever  NUMOUT  is  greater  than  zero. 

S 

Print  the  interindustry  transaction  results  (specifically,  the  P,  XT, 
YF,  and  ZX  matrices  in  constant  and  current  dollars). 

T 

Print  the  input-output  coefficients  and  import  shares. 

U 

Print  all  endogenous  variables. 

V 

Print  all  exogenous  and  lagged  variables. 

W 

Print  all  coefficients. 

X 

Diagnostic  messages  in  subroutine  GETOUT. 

Z 

Writes  the  simulated  values  of  NUMOUT  variables  to  logical  unit  12 
(the  values  are  written  in  card  image  format). 
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To  get  around  the  first  problem,  a new  subroutine,  CHROPT,  was  developed. 
Subroutine  CHROPT  initializes  an  array,  FLDTA,  in  terms  of  the  host 
installation’s  FORTRAN  character  set.  To  get  around  the  second  problem,  an 
explicit  CHARACTER  statement  is  used  rather  than  word  bytes  (i.e.,  INTEGER 
declarations) . 

The  minor  revisions  to  the  DGEM  source  code  will  now  be  described.  The 
DGEM  data  file  consists  of  19,327  entries  for  each  of  43  years  (i.e.,  1958 
through  the  year  2000).  Consequently,  it  was  necessary  to  properly  define  the 
file  so  that  individual  entries  could  be  retrieved  and/or  stored.  This  was 
accomplished  through  the  use  of  OPEN  statements  in  the  MAIN  program  for  the 
Cyber  version  and  in  the  DATA  and  SETDAT  subroutines  for  the  VAX  version.  One 
reason  for  the  difference  in  where  the  OPEN  statements  are  placed  has  to  do  with 
whether  the  operating;  system  can  handle  large  direct-access  files. 

Several  subroutines  in  the  original  DGEM  source  code  used  DATA  statements 
to  initialize  key  variables.  For  example,  in  subroutine  NIPABR  the  variables 
TEMPX  and  TEMPM  are  used  to  calculate  for  each  year  of  the  simulation  the  value 
of  exports  and  imports,  respectively.  If  the  values  are  initially  set  to  zero 
with  a DATA  statement,  then  with  each  subsequent  call  to  subroutine  NIPABR  they 
will  contain  the  cumrnulative  values  for  all  previous  years  of  exports  and 
imports.  Consequently,  it  was  necessary  to  replace  all  such  DATA  statements 
with  explicit  statements,  such  as 

TEMPX=0.0 

TEMPM =0.0  , 


in  each  of  the  affected  subroutines. 

Based  on  discussions  with  the  model  developers,  it  was  decided  to  change 
the  settings  for  the  variables  used  to  test  for  convergence.  These  changes  were 
all  within  the  Newton’s  method  subroutines  (i.e.,  NEWTM , NEWTMA,  NEWTMB);  in 
each  case  the  stringency  of  the  convergence  criterion  was  relaxed  somewhat.  The 
effect  of  these  changes  was  to  significantly  speed  up  convergence,  because  fewer 
iterations  were  required,  without  significantly  changing  the  precision  of  the 
solution  values. 

Several  other  changes  which  should  be  noted  involve  the  insertion  of 
explicit  COMMENT  statements  and  the  addition  of  two  new  subroutines.  The  first 
change  was  performed  to  replace  within  line  comments  with  full  line  comments. 
Where  appropriate,  additional  comment  lines  were  inserted  to  provide  greater 
detail  on  what  calculations  were  being  performed.  The  two  new  subroutines  were 
required  to  accommodate  the  new  capabilities  provided  through  the  use  of  the 
Mobilization  TUTOR.  The  two  new  subroutines  are  CONVT72  and  CONVT82,  which 
convert  government  purchases  and  exports  to  1972  and  1982  dollars, 
respectively. 
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7.2.2  The  Mobilization  TUTOR 


In  an  actual  application,  it  is  necessary  for  the  analyst  to  first  construct 
a runstream  file.  This  file  defines  the  years  of  the  simulation,  the  variables 
and  their  values  which  require  modification,  and  the  types  of  reports  desired. 

The  runstream  file  consists  of  two  parts.  The  first  is  a set  of  job 
control  language  to  set  up  the  file  assignments  and  initiate  execution.  The 
second  is  a set  of  control  instructions. 

The  control  instructions  consist  of  two  parts.  The  first  part  of  the 
control  instructions  contains  the  following  information: 

1.  The  options  selected  by  the  analyst  for  specialized  outputs  and  file 
manipulations ; 


2.  The  period  of  the  simulation; 


3.  Whether  the  simulation  is  static 

4.  The  number  of  variables  to  be  sea 


5.  The  number  of 

6.  The  number  of 
instrument  pa 

7.  The  number  of 

The  second  part 
information: 


variables  to  be  set 

endogenous  variable 
irs)  that  the  analys 

reports  for  variabl 

of  the  control  inst 


or  dynamic; 

led  under  instructions  from  the  analyst; 

to  specified  values  by  the  analyst; 

equality  constraints  (i.e.,  target/ 
t is  going  to  specify;  and 

es  that  the  analyst  is  going  to  specify. 

ructions  contains  the  following 


1.  Names  of  exogenous  variables  or  coefficients  to  be  scaled; 

2.  Scale  factors  for  each  of  the  named  exogenous  variables  or  coefficients; 

3.  Names  of  exogenous  variables  or  coefficients  to  be  set; 

4.  Numerical  values,  for  each  year  of  the  simulations,  for  each  of  the  named 
exogenous  variables  or  coefficients  to  be  set  equal  to; 

5.  Pairs  of  variable  names,  one  pair  for  each  endogenous  variable  to  be 
targeted ; 


6.  Numerical  values  for  the  targets,  for  each  year  of  the  simulations,  for 
each  of  the  endogenous  variables  to  be  constrained;  and 

7.  Names  of  variables  for  which  detailed  reports  are  to  be  generated. 

The  previous  discussion  provides  an  indication  of  both  the  complexities  and 
subtleties  inherent  in  the  construction  of  the  runstream  file.  Consequently,  in 
order  to  promote  the  use  of  DGEM  by  a wide  class  of  analysts,  an  on-line  TUTOR 
was  developed. 
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The  TUTOR,  through  a nested  set  of  menus,  prompts  the  analyst  to  provide 
information  which  captures  the  economic  issues  inherent  in  major  application 
areas  such  as  military  mobilization.  Furthermore,  the  TUTOR  provides  direct, 
on-line  access  to  the  data  file.  The  analyst  in  constructing  the  runstream  can 
therefore  examine  the  base-case  values  (i.e.,  both  the  realized  and  projected) 
contained  in  the  data  file  to  provide  guidance  in  selecting  the  values  of  key 
variables  for  the  event  to  be  analyzed.  In  cases  where  endogenous  variables  are 
to  be  set  to  a specified  level  (i.e.,  targeted),  the  TUTOR  recommends  to  the 
analyst  one  or  more  instruments  which  will  affect  the  value  of  the  targeted 
variable.  The  TUTOR  stores  all  intermediate  results  in  a scratch  file.  Once  the 
analyst  is  satisfied  that  answers  to  all  of  the  major  questions  have  been 
provided,  the  TUTOR  constructs  and  formats  a runstream  file.  The  TUTOR-genera ted 
runstream  file  can  be  used  to  execute  DGEM  or  further  refined  by  the  analyst. 

Initially,  the  TUTOR  presents  the  analyst  with  several  areas  or  options  each 
characterizing  some  aspect  of  the  event.  This  is  the  start  of  the  detailed 
specification  of  the  event.  Indicators  of  the  industry,  structure  and  duration 
of  the  event  (e.g.,  military  mobilization)  are  selected  or  specified  here. 

Policy  responses  are  also  selected  here,  as  are  reports  on  the  economic  effects 
of  the  event. 

The  definition  of  the  event  is  accomplished  through  a tree  structure.  Each 
area  of  economic  or  policy  specification  leads,  according  to  which  instruction 
the  analyst  provides,  to  more  detailed  areas.  In  this  way,  the  analyst  can 
provide  a comprehensive  definition  of  the  event  in  areas  of  greatest  concern, 
while  providing  only  the  most  general  definition  in  other  areas.  In  other  words, 
the  analyst  can  control  the  definition  of  the  event  to  be  analyzed. 

There  are  four  broad  areas  that  provide  the  starting  points  for  the 
definition  of  the  event. 


1. 

Defense  and 

other  government  purchases 

2. 

Resources  and  foreign  trade. 

3. 

Government 

policies . 

4. 

Reports . 

Defense  and  Other  Government  Purchases 

Government  purchases  are  separated  into  three  types  of  government  activity: 
Federal  defense;  Federal  nondefense;  and  state  and  local  government.  In  turn, 
each  type  of  government  activity  involves  purchases,  spread  over  a number  of 
supplying  industries.  For  Federal  defense  purchases,  a rudimentary  bridge  table, 
which  distributes  defense  purchases  from  the  55  DoD  procurement  categories  to  the 
36  producing  sectors,  is  included  in  the  TUTOR.  The  analyst  can  therefore 
control  defense  spending  in  a variety  of  ways  within  this  structure.  Overall 
levels  of  defense  spending  can  be  altered,  leaving  the  composition  unchanged,  or 
the  composition  and  level  can  be  changed.  Specific  defense  procurement 
categories  can  be  altered,  year  by  year.  Federal  nondefense  purchases  and  state 
and  local  government  purchases  are  spread  over  the  36  producing  sectors.  Federal 
nondefense  purchases  can  also  be  altered  in  total  or  industry  by  industry.  State 
and  local  purchases  can  be  altered  in  total,  although  the  composition  remains  the 


same . 
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Resources  and  Foreign  Trade 


This  area  covers  critical  inputs  to  production.  Generally,  labor  and 
capital  are  the  most  important  inputs.  Labor  covers  overall  population  and  labor 
conditions  as  well  as  the  availability  of  labor  to  specific  industries. 

Similarly,  overall  capital  stock  and  effective  capital  input  are  covered,  as  well 
as  capital  availability  to  specific  industries.  Foreign  trade  covers  imports  and 
exports,  each  of  which  can  be  affected  by  an  event. 

A detailed  list  of  indicators  describing  population  and  labor  input  is 
provided  should  the  analyst  choose  to  go  through  the  labor  branches  of  the 
specification  tree.  Initial  population,  population  growth  rates,  labor  force 
participation  patterns,  work  hours  and  vacation  patterns,  and  labor  efficiency 
are  all  included.  Also,  the  analyst  can  specify  how  labor  is  used  (i.e.,  the 
allocation  of  labor  between  industries  and  the  amount  of  labor  available  to 
particular  industries). 

Similar  possibilities  are  included  for  capital.  The  initial  operational 
capital  stock,  the  efficiency  or  intensity  of  use  of  the  capital  stock,  and 
depreciation  rates  describe  the  general  features  of  capital.  Finally,  capital 
availability  by  industry  can  be  specified.  For  example,  if  it  were  known  that 
certain  industries  had  capacity  reserves  which  could  be  called  upon  in  times  of 
emergency,  these  net  additions  to  the  capital  stock  could  be  modeled  explicitly 
by  the  analyst. 

Imports  can  be  specified  industry  by  industry  and  the  analyst  also  has  the 
choice  of  tariffs  or  quantitative  controls  for  limiting  imports.  Exports  can  be 
altered  industry  by  industry  or  the  overall  volume  of  exports  can  be  changed, 
keeping  the  composition  unaltered.  Also,  there  are  considerations  for  the 
balance  of  trade.  This  can  be  unconstrained  or  it  can  be  subject  to  limits, 
reflecting  the  constraints  that  usually  exist  on  the  balance  of  trade. 

Government  Policies 

A wide  range  of  policy  measures  are  available  to  the  U.S.  government  for  use 
during  or  after  a major  event  such  as  military  mobilization.  Some  of  these 
policies  involve  purchases  of  goods  and  services;  some  policies  control  the 
allocation  and  use  of  available  resources,  in  particular  labor  and  capital;  some 
policies  control  production  activities,  the  level  of  operation  of  specific 
industries;  some  policies  involve  monetary  and  credit  conditions,  affecting  the 
level  of  borrowing  and  spending;  and  some  policies  affect  fiscal  conditions,  such 
as  government  spending,  taxation,  deficits  and  borrowing.  These  types  of 
policies  are  included  and  can  be  specified  by  the  analyst. 

For  example,  fiscal  policy  involves  government  revenues  and  expenditures, 
and  their  difference,  the  government  deficit.  Government  revenues,  primarily 
taxes,  reduce  the  level  of  private  disposable  income  and  spending.  Government 
purchases  are  themselves  an  element  of  spending  and  directly  add  to  private 
spending  in  demand  for  goods  and  services.  The  government  deficit  affects  the 
capital  markets,  since  the  deficit  is  financed  by  government  borrowing.  The 
larger  the  deficit,  the  more  private  capital  is  preempted  (i.e.,  crowded  out), 
reducing  the  availability  of  funds  for  private  spending,  and  showing  up 
particularly  in  its  effect  on  private  investment.  Each  of  these  policy  measures 
is  available  to  the  government  and  each  can  be  adjusted  by  the  analyst  in 
designing  the  analysis  of  the  event. 
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Reports 


The  analyst  can  specify  what  information  to  retrieve  from  the  simulation. 
While  all  of  the  previous  considerations  referred  to  defining  the  event  to  be 
investigated,  this  area  covers  the  data  to  be  reported  after  the  simulation  has 
been  completed.  A range  of  reports  is  available  and  the  analyst  can  select  from 
these  to  tailor  the  computer  reports  to  the  types  of  indicators  and  analysis 
desired . 

The  first  set  of  reports  covers  the  Gross  National  Product  accounts.  These 
can  be  reported  in  current  dollars,  in  constant  dollars,  or  in  terms  of  price 
indices.  They  can  also  be  reported  in  terms  of  different  types  of  spending 
(i.e.,  consumption,  investment,  government,  exports  and  imports).  At  the 
aggregate  level,  the  Gross  National  Product  indicators  summarize  spending, 
production  and  prices. 

Government  purchases  are  covered  in  the  next  set  of  reports.  Then,  there 
are  reports  on  labor.  These  cover  population,  labor  supply,  labor  prices  (or 
compensation  rates),  and  labor  input  by  industry.  Similarly,  for  capital,  there 
are  reports  on  capital  stock,  capital  input,  rates  of  return  and  capital  input  by 
industry.  At  the  industry  level,  there  are  reports  on  production  levels  and  on 
market  demands.  Prices  can  be  reported  at  the  overall  level,  by  good  or  service, 
and  by  industry.  Interest  rates  can  also  be  requested.  All  of  these  choices  are 

menu-driven.  In  addition,  there  is  opportunity  for  the  analyst  to  request  a 

report  on  any  variable  in  the  model  by  specifying  the  name  of  the  variable. 

7.3  DGEM  FROM  THE  USER’S  POINT  OF  VIEW 

Over  the  past  year,  DGEM  has  been  used  extensively  by  the  NBS  project  team. 
The  model  is  also  currently  being  used  by  at  least  five  Federal  agencies  and  four 

non-Federal  installations.  Whereas  the  work  carried  out  at  NBS  has  encompassed  a 

wide  variety  of  modeling  techniques,  the  activities  of  the  other  users  have  had  a 
narrower  focus.  In  particular,  much  of  the  non-NBS  work  has  focused  on  gaining  a 
better  understanding  of  the  model  through  interactive  sessions  on  the  TUTOR. 

The  installations  to  which  NBS  has  supplied  copies  of  the  DGEM  software 
system-*-  include: 

(1)  Economic  Research  Service 

U.S.  Department  of  Agriculture; 

(2)  Office  of  Business  Analysis 
U.S.  Department  of  Commerce; 

(3)  Office  of  Industrial  Base  Assessment 
U.S.  Department  of  Defense; 

(4)  Office  of  Industrial  Mobilization  Programs 
General  Research  Corporation; 


1 Whenever  the  term  DGEM  software  system  is  used,  it  is  meant  to  include  the 

source  code  for  both  DGEM  and  the  TUTOR,  the  DGEM  database,  and  a set  of  one  or 
more  test  problems  and  solutions.  Software  documentation  is  also  provided. 
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(5)  Systems  Analysis  Department 

Swedish  National  Defense  Research  Institute; 

(6)  Department  of  Operations  Research  and  Applied  Statistics 
George  Mason  University. 

All  three  of  the  non-Federal  installations  were  provided  with  VAX  versions 
of  the  model.  A brief  description  of  the  activities  carried  out  at  each  of  these 
installations  is  given  in  the  text  which  follows.  The  three  Federal 
installations  will  then  be  discussed. 

The  DGEM  software  system  was  provided  to  the  General  Research  Corporation  as 
a potential  candidate  for  inclusion  in  a software  package  being  developed  for  the 
Army  Material  Command  (AMC).  Since  the  target  delivery  system  for  the  AMC 
software  package  was  a PC-AT  class  machine,  the  role  of  DGEM  was  quite  limited  in 
this  application.^ 

The  Swedish  National  Defense  Research  Institute  is  currently  pursuing  a 
research  project  on  the  economic  impacts  of  military  mobilization.  Their 
interest  in  DGEM  is  primarily  pedagogical  because  DGEM  is  calibrated  with  U.S. 
data.  However,  because  DGEM  explicitly  allows  for  major  swings  in  both  labor 
services  and  international  trade,  a DGEM-like  model  calibrated  with  Swedish  data 
may  provide  an  analytical  tool  which  is  currently  unavailable  to  Swedish  defense 
planners . 

The  activities  carried  out  at  George  Mason  University  focused  on  how  to 
perform  a structured  sensitivity  analysis  with  the  model.  Previous  work  by  one 
of  the  authors  (Harris),  produced  a set  of  stand  alone  software  for  creating 
random  Latin  hypercube  sampling  schemes.  Input  from  experts  on  military 
mobilization  was  then  used  to  identify  a set  of  key  decision  variables  which 
would  strongly  affect  the  path  of  the  economy  over  the  mobilization  period. 

These  variables  were  then  run  through  the  stand  alone  software  package  to 
establish  an  experimental  design  for  executing  DGEM.  In  the  interim,  the  DGEM 
software  system  was  setup  and  tested  on  the  George  Mason  University  VAX.  The 
TUTOR  was  first  used  to  construct  a base-case  simulation  runstream  for  the  period 
of  interest;  a mobilization  baseline  (see  Section  9.2.1)  was  then  constructed. 
Both  runstreams  were  executed  and  compared  to  results  produced  on  the  NBS 
computer  systems.  Individual  runstream  files  for  the  structured  sensitivity 
analysis  were  then  constructed  via  the  VAX  editor.  This  approach  was  taken 
because  it  was  considerably  faster  to  copy  and  then  edit  the  runstream  files  to 
comply  with  the  experimental  design  than  to  create  them  directly  via  the  TUTOR. 
The  results  of  these  and  other  activities  are  summarized  in  Chapters  9 and  10. 

The  three  Federal  agencies  to  which  NBS  provided  copies  of  DGEM  are  applying 
it  in  two  distinct  modes  of  operation.  The  Economic  Research  Service  is 
indicative  of  the  first  mode  of  operation;  they  view  DGEM  as  a tool  which  is 
complementary  to  certain  key  models  used  by  or  within  their  agency.  For  example, 
forecasts  produced  by  DGEM  can  be  compared  to  those  of  proprietary  macroeconomic 
models  to  identify  areas  where  refinements  to  the  problem  under  analysis  may  be 
called  for.  Similarly,  the  forecast  values  produced  by  DGEM  may  be  used  as 
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Subsequent  work  by  FEMA  has  produced  a PC-based  version  of  DGEM. 


inputs  for  certain  key  variables  in  large-scale  models  such  as  the  Economic 
Research  Service's  Food  and  Agriculture  Policy  Simulator  (FAPSIM).  The  Office  of 
Business  Analysis  and  the  Office  of  Industrial  Base  Assessment  are  indicative  of 
the  second  mode  of  operation;  their  focus  is  on  major  emergencies  (e.g.,  military 
mobilization)  and  their  impacts  on  national  security.  In  this  mode,  DGEM  is  part 
of  a larger  system  which  integrates  information  from  other  models  and  databases. 
Although  the  applications  envisioned  by  the  Office  of  Business  Analysis  and  the 
Office  of  Industrial  Base  Assessment  are  in  accordance  with  the  stated  "analysis 
strategy"  of  DGEM,  the  degree  of  detail  implied  in  these  applications  is  much 
finer  than  currently  exists  within  DGEM.  This  leads  one  to  conclude  that  if  DGEM 
is  to  be  better  integrated  into  these  "analytical  systems"  it  will  first  be 
necessary  to  construct  several  bridge  tables  which  disaggregate  the  forecast 
values  of  selected  DGEM  variables  to  agree  with  those  used  elsewhere  in  the 
system. 

All  three  Federal  agencies  have  found  the  TUTOR  to  be  of  great  assistance  in 
understanding  how  the  model  operates,  how  the  "scenario"  of  interest  may  be 
defined,  and  what  to  expect  from  the  model  in  terms  of  output.  Because  the 
analyst  can  "callup"  the  values  of  key  variables  via  the  TUTOR,  it  provides 
significant  insights  into  the  assumptions  underlying  the  base-case  simulation. 
Furthermore,  since  one  of  the  most  complicated  parts  of  the  software  transfer  is 
the  creation  of  the  DGEM  database  as  a mass  storage  file  from  the  sequential  file 
on  the  tape  provided,  the  relatively  straight  forward  programming  style  within 
the  TUTOR  facilitates  the  process  of  properly  defining  the  structure  of  the  DGEM 
database  on  the  host  system. 

The  previous  discussion  does  not  reflect  the  considerable  use  made  of  DGEM 
by  FE^IA,  its  developer,  Dale  Jorgenson  and  Associates  (DJA) , and  by  several  other 
organizations.  DGEM  has  been  used  by  FEMA  to  simulate  a number  of  major 
emergency  situations.  One  subject  which  is  of  great  importance  to  FEMA  is  that 
DGEM  is  the  only  macroeconomic  model  with  demonstrated  capability  to  simulate  the 
U.S.  economy  following  a nuclear  attack.  In  addition,  FEMA's  experience  with 
DGEM  indicates  that  there  appears  to  be  no  identifiable  limit  to  the  percentage 
increase  or  decrease  in  economic  activity  that  DGEM  can  simulate  if  handled  in 
stepwise  increments,  an  attribute  thought  to  be  unique  among  macroeconomic 
models.  DGEM  has  been  employed  extensively  by  DJA  on  behalf  of  FEMA,  the 
Department  of  Energy,  the  Congress  and  other  organizations.  DGEM  is  currently  in 
use  at  the  Oak  Ridge  National  Laboratory  and  was  used  extensively  by  the  Brook- 
haven  National  Laboratory  in  support  of  the  Department  of  Energy.  DGEM  was 
recently  employed  as  a macroeconomic  driver  of  the  Transportation  Express  Model 
used  by  the  Department  of  Transportation  in  the  training  of  National  Defense 
Executive  Reservists. 

By  and  large,  the  NBS  experience  with  DGEM  has  been  quite  positive.  The 
relative  completeness  of  the  software  documentation  simplified  our  initial 
efforts  at  setting  up  the  model  on  the  original  UNIVAC  1100-82  computer  system. 
The  code  conversion  which  followed  immediately  thereafter  and  the  development  of 
the  TUTOR  produced  a deeper  understanding  of  the  model's  capabilities  and 
limitations.  The  success  in  moving  between  computer  systems  and  being  able  to 
replicate  results  was  also  quite  encouraging. 
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Based  on  our  work  with  the  model  and  feedback  from  other  parties,  it  is 
clear  that  DGEM  can  produce  information  which  provides  significant  insights  into 
a wide-variety  of  policy  issues.  However,  one  should  recognize  that  some  of 
these  advantages  may  be  adversely  impacted  if  nothing  is  done  to  remedy  two 
deficiencies.  These  deficiencies,  as  perceived  both  by  the  NBS  project  team  and 
other  informed  parties,  are  the  lack  of  a software  maintenance  plan  and  the 
highly  aggregated  nature  of  the  DGEM  solution. 

The  need  for  a software  maintenance  plan  has  become  increasingly  important 
over  the  past  year.  This  need  stems  from  three  interdependent  sources; 


(1)  the  development  of  the  TUTOR; 

% 

(2)  the  incorporation  of  monetary  policy  into  the  model;  and 

(3)  the  requirement  to  revise  portions  of  the  database  to  accommodate  the 
previously-mentioned  enhancements . 

To  a certain  extent  the  current  "clearinghouse"  role  of  NBS  has  enabled 
users  to  obtain  the  most  up-to-date  information  on  the  model.  'However,  this  role 
needs  to  be  formalized  to  ensure  that  all  users  have  the  benefit  of  the  latest 
software  enhancements,  as  well  as  revisions  to  the  database  and  documentation. 


The  need  for  increased  sectoral  detail  in  DGEM  has  been  recognized  by  a 
number  of  modelers  for  sometime.  Although  this  is  a serious  limitation  of  the 
model  for  some  applications,  it  may  cause  little  or  no  problem  for  others.  In 
areas  where  a lack  of  sectoral  detail  is  a problem,  it  may  be  possible  to  employ 
bridge  tables  as  a solution.  Two  areas  where  this  approach  appears  feasible  are: 
(1)  a bridge  between  the  various  military  procurement  categories  and  the  DGEM 
vector  of  final  demands;  and  (2)  a bridge  between  the  DGEM  solution  and  the 
sectors  defined  by  the  Commerce  Department’s  4-digit  Standard  Industrial 
Classification  (SIC).  Whereas  the  construction  of  the  first  bridge  table  is 
straightforward,-*-  the  second  may  require  the  redefinition  of  some  internal 
variables  and  entail  a restructuring  of  the  source  code.  Both  sets  of  bridge 
tables  should  be  studied  by  a DGEM  user  group  to  define  their  use  and  assess 
their  technical  merits. 


1 Such  a bridge  table  could  be  incorporated  directly  into  the  TUTOR.  Data  for 
constructing  the  bridge  table  exists  both  within  the  Defense  Department  and  the 
Commerce  Department  (see  Appendix  D). 
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THE  DGEM  DATA  BASE 


The  DGEM  database  consists  of  19,327  entries  for  each  of  43  years  (i.e., 

1958  through  the  year  2000).  Of  the  19,327  entries  for  each  year  5,903  are 
endogenous  variables,  1,643  are  exogenous  variables,  and  11,781  are  coefficients. 
The  estimation  of  the  model  over  the  historical  period  (i.e.,  1958  to  1974)  used 
a database  constructed  by  Jack  Faucett  and  Associates. ^ The  Faucett  database 
contains  annual  price  and  quantity  data  on  11  types  of  energy  and  over  30 
intermediate  inputs  for  35  producing  sectors  of  the  U.S.  economy  over  the  years 
1958  through  1974. 

Since  work  began  on  DGEM  in  the  mid  1970’s,  several  studies  have  examined 
the  Faucett  database.  Two  studies  which  are  particularly  noteworthy  for  their 
thoroughness  and  methodological  approach  are  due  to  Berndt,  Fuss  and  Waverman^ 
and  Hazilla  and  Kopp.^  The  latter  study  provides  many  insights  into  the  Faucett 
database  because  it  examines  the  nature  of  factor  adjustment,  the  treatment  of 
capital  aggregation,  alternative  capital  service  price  formulations,  the 
aggregation  of  energy  types,  the  specification  of  technological  change,  and  the 
choice  of  scale  economies.  One  conclusion  from  the  Hazilla  and  Kopp  study  which 
is  of  importance  to  the  DGEM  database  is  that  substitution  elasticity  estimates 
are  quite  sensitive  to  the  capital  service  price  formulation.  In  a subsequent 
review  of  the  Hazilla  and  Kopp  study,  Berndt  points  out  that  the  rate  of  return 
used  in  DGEM  is  based  on  realized  profits  and  taxes,  and  thus  is  an  ex 
post  rather  than  an  ex  ante  rate  of  return. ^,5  This  poses  a methodological 
problem  because  the  choice  of  the  capital  service  price  formulation  confuses  ex 
ante  with  ex  post  costs  of  capital.  Berndt  asserts  that  if  DGEM  were  estimated 
using  the  theoretically  appropriate  ex  ante  cost  of  capital  (e.g.,  the  Moody  Baa 
bond  yield  which  captures  average  risk),  that  the  new  parameter  and  elasticity 
estimates  would  differ  considerably  from  those  now  found  in  the  DGEM  database. 

As  a part  of  the  original  task  of  building  and  testing  DGEM,  the  developers 
prepared  a number  of  base-case  simulations.  Three  sets  of  base-case  simulations 
are  reviewed  in  this  section;  they  are: 


Jack  Faucett  and  Associates,  Development  of  35  Order  Input-Output  Tables, 
1958-1974,  Final  Report,  Washington,  DC:  Federal  Preparedness  Agency,  1977. 

^ Berndt,  E.R.,  M.  A.  Fuss  and  L.  Waverman,  Empirical  Analysis  of  Dynamic 

Adjustment  Models  of  the  Demand  for  Energy  in  U.S.  Manufacturing  Industries, 
1 947-1974 , Palo  Alto,  CA:  Electric  Power  Research  Institute,  1980. 

^ Hazilla,  M.,  and  R.  J.  Kopp,  Industrial  Energy  Substitution:  Econometric 

Analysis  of  U.S.  Data,  1958-1974,  Palo  Alto,  CA:  Electric  Power  Research 

Institute,  1984. 

^ Berndt,  E.  R. , "Summary/Critique  of  Industrial  Energy  Substitution: 
Econometric  Analysis  of  U.S.  Data,  1958-1974,"  mimeo,  1985. 

Since  the  ex  post  rate  of  return  is  used  within  DGEM  as  the  ex  ante  cost  of 
capital,  one  would  conclude  that  the  firm  is  always  in  equilibrium. 
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(1)  model  performance  over  the  historical  period  (i.e.,  simulations 
between  1959  and  1974); 

(2)  model  performance  between  1975  and  1981,  when  published  data  on 
certain  key  variables  was  available;  and 

(3)  the  base-case  projections  (i.e.,  simulations  between  1982  and  the 
year  2000). 

The  historical  simulations  were  of  two  types,  static  and  dynamic.  Both 
types  of  simulation  were  compared  to  the  realized  values  of  personal  consumption 
expenditures,  gross  private  domestic  investment  and  GNP  from  the  National  Income 
and  Product  Accounts  (NIPA).^-  All  comparisons  used  as  a goodness-of-f it  measure 
the  root-mean-squared  error  (RMSE)  as  a percent  of  the  actual  mean. 

The  goodness-of-f it  measures  associated  with  the  three  NIPA  components  for 
both  the  static  and  dynamic  simulations  are  recorded  in  Table  8.1.  For  the 
static  simulation,  the  maximum  RMSE  is  8.4  percent  for  nominal  investment  and 
11.9  percent  for  real  investment  (calculated  in  1972  dollars).  The  maximum  and 
minimum  RMSE  figures  for  the  sectoral  prices  are  14.7  percent  for  Tobacco 
Manufacturers  and  2.0  percent  for  Gas  Utilities.  The  corresponding  figures  for 
sectoral  quantities  are  49.5  percent  for  Tobacco  Manufacturers  and  3.0  percent 
for  Transportation  Equipment  and  Ordnance,  except  Motor  Vehicles. 

For  the  dynamic  simulation,  the  maximum  RMSE  is  12.2  percent  for  nominal 
investment  and  24.0  percent  for  real  investment.  The  maximum  and  minimum  RMSE 
figures  for  the  sectoral  prices  are  58.7  percent  for  Tobacco  Manufacturers  and 
3.7  percent  for  Primary  Metal  Industries.  The  maximum  and  minimum  RMSE  for  the 
sectoral  quantities  is  455.2  percent  for  Tobacco  Manufacturers  and  3.1  percent 
for  Transportation  Equipment  and  Ordnance,  except  Motor  Vehicles.  Other  dynamic 
simulations  were  performed  within  the  historical  period  which  did  not  include  the 
years  prior  to  I960.  These  simulations  produced  significantly  better  agreement 
with  published  data. 

In  evaluating  DGEM's  performance  between  1975  and  1981,  one  must  recognize 
that  while  many  variables  were  aligned  to  observed  values,  many  were  not.  In 
most  cases,  the  simulated  values  for  these  compare  favorably  with  information 
developed  from  published  sources.  The  discussion  which  follows  provides  several 
specific  examples  from  which  the  reader  can  gauge  the  model’s  performance. 


^ Simulations  of  the  36  DGEM  Over  the  Historical  Period,  Lexington,  MA:  Data 

Resources,  Inc.,  198U. 
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Table  8.1  Goodness-of-Fi t Measures  for  Three  NIPA  Components  During  the 
Historical  Period-*- 


NIPA 

Static 

Dynamic 

Component 

Nomi nal 

Real^ 

Nomi nal 

Real  ^ 

Personal  Consumption 

1.8 

2.7 

6.2 

9.2 

Investment 

'Cf- 

. 

00 

11.9 

12.2 

24.0 

GNP 

4.0 

4.1 

3.2 

10.7 

Between  1975  and  1981,  actual  inflation  exceeded  that  predicted  by  the 
model.  The  model  also  underestimates  inflation  in  export  and  import  prices,  but 
it  accurately  tracks  the  relative  price  trend  which  is  important  for  considering 
exchange  rate  movements. 

The  underestimate  of  ouput  inflation  has  its  origins  with  projected  growth 
in  input  prices.  The  price  of  labor  received  by  households,  PLB,  was  used  as  the 
instrument  for  nominal  consumption,  CNIA.  Simulated  growth  in  PLB  averaged  8.3 
percent  annually  from  1974  through  1981.  This  compares  with  a 9.0  percent  annual 
rate  for  hourly  labor  compensation  in  the  nonfarm  business  sector  over  the  same 
period.  With  broadly,  though  not  exactly,  similar  coverage  of  the  labor  market, 
the  model  predicts  somewhat  slower  growth  in  labor  costs  which  has  a 
corresponding  impact  on  output  prices.  Growth  in  capital  prices  is  in  close 
agreement  with  past  trends. 

The  model  slightly  overestimates  income  growth.  Over  the  1974  through  1981 
period,  national  income  (in  current  dollars)  grew  at  an  average  annual  rate  of 
10. b percent.  The  nominal  income  variable  in  DGEM , Y,  rose  moderately  faster,  at 
an  average  rate  of  11.6  percent  annually.  This  combined  with  lower  inflation 
implies  more  rapid  growth  in  real  incomes  within  DGEM  than  did  actually  occur. 
Since  spending  is  aligned  to  observed  history,  the  balance  is  saved,  adding  to 
private  wealth.  The  mechanisms  that  promote  this  additional  saving  are  the 
after-tax  rates  of  return  on  capital  and  wealth. 


-*-  All  entries  are  the  RMSE  as  a percent  of  the  actual  mean. 
^ Calculations  are  based  on  1972  dollars. 
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A final  area  for  comparison  concerns  economic  structure.  Periodically,  the 
U.S.  Department  of  Labor  publishes  time-series  data  on  domestic  output  for  all 
input-output  industries.  These  data  can  be  aggregated  to  conform  to  the 
interindustry  structure  represented  in  DGEM . Table  8.2  compares  the  Labor  and 
DGEM  data  for  the  years  1974  and  1980.  Data  from  the  model  for  1974  are  as 
developed  by  Jack  Faucett  and  Associates,  while  the  data  for  1980  are  the  results 
of  simulations.  Reference  to  Table  8.2  shows  close  agreement  in  the  relative 
importance  of  each  industry  and  the  changing  nature  of  that  importance  over  time. 
For  example,  the  shift  away  from  construction  and  the  so-called  "smokestack” 
industries  (e.g.,  primary  metals,  motor  vehicles)  and  toward  communications, 
trade,  finance,  and  services  is  captured  in  the  simulation. 

The  base-case  projection  represents  a simulated  path  of  economic  growth  and 
structural  change  from  1982  to  the  year  2000.-*-  The  projection  reflects  both  the 
assumed  values  of  the  exogenous  variables  and  the  structure  of  DGEM  itself.  The 
exogenous  variables  values  reflect  best  estimates  incorporating  information 
available  in  1983.  Correspondingly,  the  base-case  projection  might  be  viewed  as 
a "best  estimate"  of  the  economic  future,  as  seen  from  the  perspective  of  1983. 

The  long  term  trends  in  real  economic  growth  over  the  18-year  period  are 
recorded  in  Table  8.3.  Economic  growth  is  projected  to  average  just  over  3 
percent  a year.  This  represents  continued  growth  at  rates  less  rapid  than  during 
the  World  War  II  to  1973  period  but  more  rapid  than  during  the  1973  to  1983 
decade. ^ There  are  two  reasons  for  this.  First,  the  1982  starting  point  was  in 
a recession,  so  the  1980's  growth  reflects  both  recovery  and  underlying  growth. 
Second,  the  rate  of  labor  growth  is  considerably  less  rapid  in  the  1990 's  than  in 
the  1980's.  All  principal  uses  of  economic  production  (e.g.,  personal 
consumption  expenditures,  investment,  and  government  purchases)  show  sustained 
growth.  However,  the  growth  rate  for  the  volume  of  government  purchases  is,  by 
assumption,  less  rapid  than  for  the  economy  as  a whole,  permitting  private 
spending  to  grow  somewhat  more  rapidly  than  GNP. 


*■  The  description  of  the  base-case  projection  follows  closely  that  given  in  R.J. 
Goettle  and  E.A.  Hudson,  Final  Report  on  the  Dynamic  General  Equilibrium  Model , 
Cambridge,  MA:  Dale  Jorgenson  and  Associates,  1984. 

The  base-case  projection  assumes  a slight  acceleration  in  inflation  from  recent 
rates,  averaging  5.3  percent  over  the  18-year  period. 
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Table  8.2  Structure  Of  Total  Domestic  Output 


1 2 


DGEM  Sector 
Number 

1974 

1980 

Labor 

DGEM 

Labor 

DGEM 

1 

4.1 

4.5 

4.5 

4.7 

2 

0.2 

0.2 

0.1 

0.2 

3 

0.3 

0.3 

0.3 

0.2 

4 

0.8 

0.7 

0.7 

0.6 

5 

0.2 

0.2 

0.2 

0.2 

6 

7.5 

7.3 

6.5 

6.1 

7 

6.2 

6.1 

6.2 

6.0 

8 

0.5 

0.5 

0.4 

0.4 

9 

1.1 

1.2 

1.2 

1.3 

. 10 

1.7 

1.9 

1.6 

1.7 

11 

lei 

1.1 

1.0 

1.0 

12 

0.5 

0.6 

0.5 

0.6 

13 

1.6 

1 . 6 

1.5 

1.5 

14 

1.5 

1.6 

1 . 6 

1.7 

15 

2.6 

2.8 

2.6 

2.8 

16 

1.7 

1 . 6 

1.7 

1.5 

17 

1.7 

1.9 

1.7 

1.9 

18 

0.3 

0.3 

0.2 

0.2 

19 

1.1 

1.1 

1.0 

1.0 

20 

3.7 

3.6 

2.5 

2.5 

21 

2.5 

2.8 

2.2 

2.5 

22 

4.1 

4.2 

4.0 

4.3 

23 

3.0 

3.2 

3.4 

3.6 

24 

3.2 

3.3 

2.7 

2.8 

25 

2.0 

2.1 

2.2 

2.1 

26 

0.9 

0.9 

0.9 

1.0 

27 

0.6 

0.7 

0 . 6 

0.6 

28 

4.3 

4.5 

3.8 

3.7 

29 

2.0 

2.1 

2.8 

3.1 

30 

1.7 

1.7 

1.9 

1.9 

31 

1.0 

0.9 

0.8 

0.8 

32 

13.5 

11.2 

13.7 

11.9 

33 

9.3 

9.1 

10.7 

10.4 

34 

12.5 

13.1 

1.0 

0.9 

35 

1.0 

1.1 

1.0 

0.9 

1 R.J.  Goettle  and  E.A.  Hudson,  Final  Report  on  the  Dynamic  General  Equilibrium 
Model , Cambridge,  MA:  Dale  Jorgenson  and  Associates,  1984. 


^ Percent  of  total  real  output  across  35  industries. 
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Table  8.3  Average  Annual  Percentage  Growth  Rates  for  Constant  Dollar  Variables 


Variable 

Time  Period 

1982-1990 

1990-2000 

1982-2000 

Consumption 

3.5 

3.1 

3.3 

Investment 

3.7 

3.9 

3.8 

Government  Purchases 

2.0 

1.5 

1.7 

Real  GNP 

3.2 

3.0 

3.1 

DGEM  explicitly  recognizes  that  economic  growth  tends  to  be  cyclical  rather 
than  steady.  Hence,  there  will  be  cycles  around  the  growth  trends  given  in  Table 
8.3.  Annual  growth  rates  for  the  same  four  variables  recorded  in  Table  8.3, 
which  incorporate  these  cycles,  are  recorded  in  Table  8.4.  The  different  types 
of  final  demand  (e.g. , personal  consumption  expenditures,  investment,  and 
government  purchases)  display  very  different  patterns  over  the  business  cycle. 
Government  purchases  are  the  most  stable  since  they  reflect  political  rather  than 
market  forces.  Although  government  purchases  may  show  uneven  growth  rates,  these 
are  not  related  to  the  business  cycle  and  generally  even  work  to  dampen  the 
magnitude  of  the  cycle.  Personal  consumption  is  intermediate.  Service  and 
nondurables  purchases  are  very  stable,  durable  purchases  are  cyclical,  so  that 
overall  consumption  volumes  show  only  a mild  cycle.  Private  investment  is  the 
most  volatile  of  the  main  spending  components.  In  recession  years,  investment 
growth  is  zero  or  even  negative;  in  the  recovery  years,  investment  volumes  grow 
much  more  rapidly  than  the  rest  of  the  economy. 

The  composition  of  final  demand  between  personal  consumption  expenditures, 
private  investment  and  government  purchases  (i.e.,  how  the  output  of  the  economy 
is  used)  is  projected  to  show  substantive  change.  Table  8.5  shows  the 
composition  of  final  demand  in  both  current  and  constant  dollar  terms. 

Perhaps  the  most  striking  feature  is  the  decline  in  the  share  of  the  final 
output  of  the  economy  devoted  to  government  purchases. ^ This  feature  is  an 
assumption  of  the  forecasts,  it  is  not  an  output  of  the  model.  The  declining 
share  of  government  leaves  a greater  percentage  of  output  available  for  the 
private  sector.  Both  consumption  and  investment  share  in  this  increase. 


1 This  refers  to  Federal,  state  and  local  government  purchases  of  goods  and 
services.  It  does  not  include  government  transfers,  a major  component  of 
government  expenditures.  These  figures  do  not  necessarily  imply  a decline  in 
the  ratio  of  government  expenditures  to  GNP. 
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Table  8.4  The  Business  Cycle:  Annual  Percentage  Growth  Rates,  1982-2000 


Year 

Variable 

Personal 

Consumption 

Private 

Investment 

Government 

Purchases 

Real 

GNP 

1983 

2.8 

5.5 

2.1 

2.2 

1984 

4.5 

7.3 

2.5 

4.4 

1985 

4.9 

5.0 

2.3 

4.5 

1986 

4.0 

-0.7 

1.9 

2.9 

1987 

2.6 

1.3 

1.7 

2.6 

1988 

3.5 

7.5 

1.8 

3.8 

1989 

2.4 

0.6 

1.8 

2.2 

1990 

3.0 

3.8 

1.8 

2.9 

1991 

2.6 

5.7 

1 e 6 

3.0 

1992 

2.5 

4.4 

1.6 

2.6 

1993 

2.0 

0.9 

1.3 

1.9 

1994 

3.9 

4.2 

1.4 

3.5 

1995 

4.1 

6.9 

1.4 

4.1 

1996 

4.1 

4.4 

1.4 

3.7 

1997 

2.7 

0.3 

1.4 

2.3 

1998 

2.8 

2.6 

1.5 

2.5 

1999 

2.9 

3.0 

1.4 

2.8 

2000 

3.8 

6.2 

1.4 

3.7 

Table  8.5  Composition  of  Final 

Demand  as 

a Percent 

of  GNP,  1982 

i 

NJ 

O 

o 
o 
1 — * 

Variable 

Current  Dollars 

Constant  Dollars 

Name 

1982 

1990 

2000 

1982 

1990 

2000 

Personal  Consumption 

64.5 

65.5 

65.7 

64.8 

66.3 

66.9 

Private  Investment 

13.8 

15*2 

16.4 

13.4 

14.0 

14.8 

Government  Purchases 

21.2 

18.9 

17.5 

19.7 

18.0 

15.7 

1 Net  exports,  a component  of  GNP , are  not  shown  separately  in  this  table. 
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However,  it  is  projected  that  the  greater  part  of  the  increase  goes  to 
consumption.  Personal  consumption  is  projected  to  rise  from  around  64  percent  of 
real  GNP  to  over  66  percent.  Investment  rises  too  but  a substantial  part  of  this 
rise  reflects  cyclical  features,  recovery  from  the  low  investment  share  in  the 
1982  starting  point.  Aside  from  cyclical  features,  the  investment  share  of  real 
GNP  increases  by  about  one  percentage  point. 

DGEM  explicitly  models  36  producing  industries,  the  output  of  each,  the 
disposition  of  this  output,  and  the  inputs  into  each.  In  fact,  the  macroeconomic 
picture  is  just  an  aggregation  of  these  industry  level  developments.  Table  8.6 
ranks  the  industries  in  terms  of  volume  growth  rates  over  the  1980's  (i.e. 
average  annual  growth  in  the  volume  or  quantity  of  sales).  These  industries 
range  from  communications,  the  fastest  growing,  at  almost  8 percent,  all  the  way 
down  to  tobacco  products,  zero  growth,  and  crude  petroleum  and  gas,  negative 
growth.  It  is  difficult  to  generalize  a pattern  as  to  which  industries  show 
rapid  growth  and  which  show  slow  growth  since  performance  depends  not  only  on 
demand  considerations  (such  as  the  demand  response  to  population  and  income 
increase)  but  also  on  supply  considerations  (covering  input  mix  and  productivity, 
operating  through  their  effect  on  prices).  However,  a rough  generalization  is  as 
follows.  The  fast  growing  industries  are  high  technology  (communications, 
instruments,  plastics,  chemicals,  electrical  machinery)  and  services 
( f inance/insurance/real  estate,  services,  trade).  The  slow  growing  industries 
are  extractive  and  resource  processing  (crude  petroleum  and  gas,  gas  utilities, 
petroleum  refining,  nonmetallic  mining,  metal  mining,  coal  mining)  and 
population-related  (tobacco  products,  leather,  agriculture,  food  products).  The 
medium  growing  industries  are  the  established  manufacturing  industries  (apparel, 
textiles,  motor  vehicles,  primary  metals,  wood  products). 

The  differences  in  growth  rates  are  substantial.  An  industry  such  as 
communications,  growing  at  7.8  percent,  compared  to  the  overall  average  growth 
rate  of  3.2  percent,  will  more  than  double  its  relative  size  over  the  1982-2000 
period;  for  example,  the  industry  provided  3 percent  of  total  gross  output  in 
1982,  but  by  the  year  2000  its  share  will  be  7.5  percent.  Or  services,  growing 
at  4 percent,  will  increase  their  relative  size  by  15  percent,  expanding  from  14 
percent  to  16  percent  of  total  gross  output.  At  the  other  end  of  the  range,  an 
industry,  such  as  tobacco  products,  showing  zero  growth  would  find  its  relative 
size  cut  by  over  40  percent.  Or,  agriculture-forestry-fisheries,  growing  at  1.7 
percent,  would  decline  in  size  from  4.5  percent  of  gross  output  in  1982  to  just 
over  3 percent  in  2000. 

These  considerations  lead  to  two  general  conclusions.  First,  there  will  be 
major  changes  in  the  industrial  mix  by  the  year  2000.  The  high  technology  and 
service  industries  will  become  more  important,  the  resource-based  industries  will 
be  relatively  smaller  than  in  1982.  Second,  even  though  major  restructuring 
will  occur,  the  structure  of  the  American  economy  in  the  year  2000  will  not,  in 
the  very  broadest  sense,  be  fundamentally  different  from  the  way  it  was  in  1982. 


75 


Table  8.6  Industry  Growth  Rates^ 


Industry 

Rate 

Rank 

Agriculture,  Forestry,  Fisheries 

1.7 

32 

Metal  Mining 

2.7 

26 

Coal  Mining 

2.6 

27 

Crude  Petroleum,  Natural  Gas 

-0.7 

36 

Nonmetallic  Mining  and  Quarrying 

2.0 

30 

Construction 

2.8 

25 

Food-Kindered  Products 

2.1 

29 

Tobacco  Manufactures 

0.0 

35 

Textile  Mill  Products 

3.0 

21 

Apparel 

2.9 

24 

Lumber  and  Wood  Products 

3.0 

22 

Furniture  and  Fixtures 

4.4 

8 

Paper  and  Allied  Products 

3.9 

12 

Printing  and  Publishing 

3.7 

14 

Chemi cals 

5.4 

4 

Petroleum  Refining 

1.2 

32 

Rubber  and  Miscellaneous  Plastic  Products 

6.3 

3 

Leather 

1.0 

33 

Stone,  Clay,  and  Glass 

3.2 

19 

Primary  Metal  Industries 

3.0 

20 

Fabricated  Metal  Products 

4.0 

11 

Machinery,  Except  Electrical 

4.0 

10 

Electrical  Machinery 

4 . 6 

7 

Motor  Vehicles  and  Equipment 
Transportation  Equipment  and  Ordanance, 

3.0 

23 

Except  Motor  Vehicles 

3.7 

15 

Instruments 

7.7 

2 

Miscellaneous  Manufacturing 

4.9 

5 

Transportation 

3.7 

13 

Communications 

7.8 

1 

Electrical  Utilities 

3.5 

17 

Gas  Utilities 

0.3 

34 

Trade 

3.6 

16 

Finance,  Insurance,  and  Real  Estate 

4.8 

6 

Services 

4.0 

9 

Government  Enterprises 

3.3 

18 

Miscellaneous 

2.2 

28 

1 Average  annual  percentage  growth  in  sales  quantity,  1982  to  1990. 
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9. 


THE  MOBILIZATION  SCENARIO 


The  mobilization  scenario  analyzed  with  DGEM  is  patterned  after  an 
unclassified  scenario  approved  by  F . C.  Ikle,  the  DoD  Under  Secretary  for 
Policy.  Basically,  the  scenario  indicates  a conventional  war  of  three  years 
duration  in  Europe,  the  Persian  Gulf,  and  Korea.  For  purposes  of  the  current  • 
study,  a one  year  warning  period  is  assumed.  The  scenario  also  provides  the 
analyst  with  considerable  latitude  in  evaluating  how  changes  in  combinations  of 
numerical  values  for  key  variables,  thus  describing  wars  of  greater  or  lesser 
intensity,  affect  the  U.S.  industrial  infrastructure. 

The  analysis  of  the  mobilization  scenario  was  carried  out  in  two  stages.  In 
the  first  stage,  a mobilization  baseline  was  constructed  which  documented  the 
departures  for  key  variables  from  the  base-case  values  (i.e. , those  contained  in 
the  data  file)  due  to  the  period  of  mobilization  and  conflict.  The  results  from 
the  first  stage  of  the  analysis  provided  information  on  gross-level  changes  in 
the  NIPA  components,  as  well  as  measures  of  the  economic  impacts  of  mobilization 
on  a sector  by  sector  basis. 

In  the  second  stage,  six  variables  were  varied  in  combination  according  to 
an  experimental  design.  The  six  variables  which  were  the  subject  of  the 
structured  sensitivity  analysis  and  their  names,  as  they  appear  in  the  tables 
which  follow,  and  in  Appendix  E are: 

1.  Federal  defense  purchases,  GZ ( 1 ) ; 

2.  Federal  nondefense  purchases,  GZ ( 2 ) ; 

3.  The  Federal  deficit,  DG; 

4.  The  efficiency  with  which  capital  is  employed,  AKD; 

5.  The  rate  of  depreciation  of  capital,  U;  and 

6.  The  effective  supply  of  labor  services,  LB. 

The  structured  sensitivity  analysis  was  based  on  Monte  Carlo  techniques.  The 
objective  here  was  to  evaluate  how  uncertainty  in  the  values  of  the  six  input 
variables  just  mentioned  translated  into  changes  in  the  level  and  composition  of 
the  NIPA  components,  capital  services  and  energy  consumption. 

9.1  SPECIFICATION  OF  THE  MOBILIZATION  BASELINE  AND  THE  STRUCTURED  SENSITIVITY 
ANALYSIS 


The  period  of  the  mobilization  follows  that  of  the  unclassified  DoD  scenario 
and  is  assumed  to  be  1983  through  1986.  The  first  year  represents  a period  of 
buildup  prior  to  the  onset  of  hostilities.  For  each  year  of  the  mobilization, 
values  for  key  economic  variables  were  specified.  Once  the  mobilization  baseline 
had  been  established,  a structured  sensitivity  analysis  was  performed.  The 
values  of  the  six  variables  which  were  the  subject  of  the  structured  sensitivity 
analysis,  their  base-case  values  and  their  values  in  the  mobilization  baseline 
are  compared  at  the  end  of  this  section.  The  specification  of  the  military 
mobilization  scenario  presented  in  this  chapter  is  as  follows: 

1.  Government  purchases; 

2.  Labor  services; 

3.  Capital  services; 
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4.  Foreign  trade; 

5.  Fiscal  policy; 

6.  Structured  sensitivity  analysis;  and 

7.  Tabular  summary. 


Government  Purchases 

Defense  purchases  are  central  to  any  mobilization  effort.  Through  the 
TUTOR,  it  is  possible  to  control  both  the  level  and  composition  of  defense 
purchases.  However,  for  the  case  at  hand,  only  the  level  is  increased  so  that 
all  defense  purchases  are  increased  in  the  same  proportion.  Defense  purchases 
are  increased  by  50  percent  in  1983,  prior  to  the  onset  of  hostilities, 
and  by  80  percent  in  1984,  120  percent  in  1985,  and  150  percent  in  1986. 

These  increases  are  relative  to  the  base  case,  or  the  economic  conditions 
in  the  absence  of  mobilization.  Federal  nondefense  purchases  are  increased  by 
25  percent  in  the  first  year  and  by  50  percent  for  the  three  following  years. 

Labor  Services 

During  the  mobilization,  many  additional  people  will  be  recruited  or 
conscripted  into  the  armed  forces.  This  is  directly  taken  into  account  through 
the  increase  in  defense  purchases.  However,  it  is  assumed  that  mobilization 
measures  will  also  be  applied  to  the  civilian  population  and  the  civilian  labor 
force.  Additional  people  are  assumed  to  be  induced  or  pressed  into  the  labor 
force.  To  reflect  this,  the  labor  force  participation  rate  is  increased  by  4 
percent  in  every  year.  Workers  are  also  assumed  to  work  longer  hours,  through 
overtime  or  just  through  an  extension  of  the  standard  workweek.  Average  hours 
per  week  are  raised  by  13  percent.  But,  these  changes  are  likely  to  adversely 
affect  labor  efficiency.  Consequently,  the  average  efficiency  of  each  hour  of 
labor  is  reduced  by  7 percent.  These  three  factors  taken  together  produce  a 9 
percent  increase  in  the  effective  supply  of  labor  services. 

Capital  Services 

A forced  increase  in  capital  use  is  assumed.  This  corresponds  to  the  more 
intensive  use  of  existing  capital,  as  plant  and  equipment  is  used  more  hours  per 
week,  and  the  inclusion  of  emergency  capacity.  The  overall  increase  .in  the 
efficiency  with  which  capital  is  employed  is  taken  to  be  5 percent.  This  figure 
is  a very  conservative  estimate  of  the  potential  for  increasing  the  efficiency  of 
the  capital  stock,  since  estimates  of  emergency  capacity  which  can  be  brought  on 
line  within  one  year  frequently  exceed  peacetime  capacity  levels  by  25  percent. 
However,  even  this  modest  increase,  is  likely  to  accelerate  the  wear  and  tear  on 
capital.  Accordingly,  the  rate  of  depreciation  of  capital  is  increased  by  10 
percent . 


Foreign  Trade 


It  is  assumed  that  all  imports  will  be  threatened, 
reduction  of  20  percent  in  imports  is  planned.  Also,  in 
domestic  requirements  are  satisfied  first,  an  across  the 
percent  in  exports  is  planned. 


An  across  the  board 
order  to  ensure  that 
board  reduction  of  50 
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Fiscal  Policy 


The  major  emphasis  here  is  the  Federal  deficit,  since  government  purchases 
have  already  been  addressed.  The  government  deficit  is  constrained  to  exceed  the 
base-case  levels  by  fixed  percentages.  Tax  rates  are  increased  in  order  to 
secure  the  additional  revenue  that  is  required  to  maintain  the  targeted  budget 
deficit  in  the  face  of  higher  federal  spending.  This  means  that  the  increase  in 
government  expenditure  is  paid  for  in  part  by  an  increase  in  taxes  and  in  part  by 
increased  deficits.  This  mixed  strategy  was  designed  to  strike  a balance  between 
increased  investment  and  inflationary  pressures.  For  example,  if  the 
mobilization  were  financed  by  taxes,  it  would  not  come  at  the  expense  of 
investment,  since  the  government  is  not  claiming  an  additional  share  of  the 
capital  market  in  order  to  finance  defense  purchases,  but  would  come  from 
consumption,  since  it  is  households  and  consumers  who  carry  the  main  burden  of 
taxes.  If,  in  contrast,  the  mobilization  were  deficit  financed,  then  the 
government  would  be  claiming  a larger  share  of  investible  funds  so  investment 
rather  than  consumption  would  likely  carry  a greater  burden  of  the  mobilization 
(i.e.,  private  investment  is  crowded  out).  For  the  case  at  hand,  the  Federal 
deficit  is  allowed  to  increase  (over  the  base-case  values)  by  50  percent  in  1983 
and  by  80  percent  in  1984  through  1986. 

Structured  Sensitivity  Analysis 

Because  the  values  of  many  key  variables  in  a mobilization  scenario  are  not 
known  with  certainty,  it  is  advisable  to  select  a small  set,  whose  impact  is 
likely  to  be  substantial,  and  subject  them  to  a structured  sensitivity  analysis. 
Variations  in  the  values  of  these  input  variables  translate  into  variations  in 
the  values  of  the  output  or  endogenous  variables  in  such  a manner  that  the 
economic  impacts  of  shocks  to  the  system  can  be  measured  quantitatively. 

The  approach  selected  for  this  study  makes  use  of  recent  work  by  McKay, 
Conover,  and  Beckman^-  and  by  Harris.- Their  work  is  based  on  the  method  of 


M.D.  McKay,  W.H.  Conover,  and  R.J.  Beckman,  "A  Comparison  of  Three  Methods  for 
Selecting  Values  of  Input  Variables  in  the  Analysis  of  Output  from  a Computer 
Code,"  Technometrics , Vol.  21,  No.  2,  1979,  pp.  239-245. 

^ C.M.  Harris,  "An  Assessment  of  Climatological  Uncertainties  Using  Monte  Carlo 
Analysis,"  in  Selected  Assessment  Strategies  Applied  to  Short-Term  Energy 
Models , Gaithersburg,  MD:  National  Bureau  of  Standards,  NBSIR  83-2672, 

1983,  pp.  30-54. 

J C.M.  Harris,  Issues  in  Sensitivity  and  Statistical  Analysis  of  Large-Scale, 
Computer-Based  Models,  Gaithersburg,  MD:  National  Bureau  of  Standards, 

GCR  84-466,  1984. 

^ C.  M.  Harris,  Computer  Generation  of  Latin  Hypercube  Sampling  Plans, 
Gaithersburg,  MD:  National  Bureau  of-  Standards,  NBS  GCR  84-476,  1985. 
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model  sampling.  The  method  of  model  sampling  is  a procedure  for  sampling  from  a 
stochastic  process  to  determine,  through  multiple  trials,  the  nature  and  effects 
of  a probability  distribution.  The  method  of  model  sampling,  or  distribution 
sampling  as  it  is  also  called,  has  a long  history  of  use  by  statisticians  to 
derive  distributions  empirically  that  are  difficult  or  impossible  to  derive  by 
other  means.  It  permits  the  effects  of  uncertainty  to  be  rigorously  analyzed. 

The  procedure  employed  in  this  study  is  known  as  the  Latin  Hypercube 
sampling  scheme.  The  procedure,  as  it?s  name  implies,  is  patterned  after  the 
classical  Latin  square.  Latin  squares  consist  of  a set  of  permutations  such  that 
a given  character  or  value  appears  only  once  in  each  row  and  each  column.  A 
Latin  hypercube  is  similar  to  a Latin  square  with  the  important  exception  that  it 
contains  more  rows  than  columns.  For  example,  if  each  column  is  thought  of  as  a 
variable  and  each  row  is  a simulation  number,  then  entries  in  the  cells  contain 
the  values  of  a set  of  equally  spaced  percentiles  from  the  parent  cumulative 
distribution  function  (CDF)  of  the  variable  of  interest.  For  the  case  at  hand, 
with  six  variables  of  interest  and  ten  simulations,  the  entries  in  the  cells  are 
the  5th,  15th,  ...,  95th  percentiles  of  the  parent  CDF. 

In  reality,  the  exact  nature  of  the  parent  CDF  (e.g.,  measures  of  central 
tendency  and  dispersion)  is  unknown.  Estimates  of  the  parameters  (e.g.,  mean  and 
variance)  of  the  CDF  can  be  made  and  uncertainty  can  be  reduced  by  investigation 
and  research.  However,  uncertainty  can  never  be  eliminated  completely  because 
new  sources  of  uncertainty  are  arising  all  the  time.  The  true  specification  of 
the  CDF  can  only  be  known  when  mobilization  is  actually  underway. 

So  that  the  flexibility  of  the  procedure  could  be  illustrated  without  undue 
attention  on  the  characterization  of  the  CDF,  it  was  decided  to  focus  on  the 
triangular  distribution.  The  triangular  distribution  was  chosen  because  it  may 
be  specified  by  three  values,  low,  median,  and  high,  which  correspond  to  the  Oth, 
50th,  and  100th  percentiles. 

Individual  values  within  each  of  the  six  triangular  distributions  were 
chosen  aaccording  to  the  Latin  hypercube  sampling  scheme  (i.e.,  the  percentiles 
recorded  in  the  cells).  These  values  were  then  recorded  in  a runstream  file. 

Each  of  the  runstream  files  were  then  executed  and  the  results  of  the  simulation 
were  stored  on-line  for  further  analysis. 

Tabular  Summary 

The  data  which  were  the  focus  of  the  structured  sensitivity  analysis  are 
summarized  in  Tables  9.1,  9.2  and  9.3.  Table  9.1  records  the  base-case  values  of 
the  six  variables  (i.e.,  as  they  appear  in  the  DGEM  data  file).  Table  9.2 
records  the  values  of  the  same  six  variables  for  the  mobilization  baseline  case. 
These  values  are  also  the  median  values  used  in  fitting  triangular  distributions 
to  each  variable.  The  minimum  and  maximum  values  for  each  of  the  six  variables 
are  recorded  in  Table  9.3.  The  information  in  Tables  9.2  and  9.3  served  to 
define  the  CDF  for  each  variable.  Values  were  then  selected  from  each  CDF 
according  to  the  Latin  hypercube  sampling  scheme. 
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Table  9.1  Base-Case  Values 


Variable 

Name 

Values  by  Year 

1983 

1984 

1985 

1986 

GZ  ( 1 ) 

207.3 

220.3 

235.7 

233.2 

GZ  ( 2) 

68.3 

72.3 

86.9 

67.62 

AKD 

0.146 

0.159 

0.160 

0.156 

U 

0.062 

0.062 

0.062 

0.062 

LB 

1037.7 

1058.3 

1050.9 

1068.9 

DG 

174.0 

167.3 

211.9 

192.5 

Table  9.2  Mobilization  Baseline  Values 


Variable 

Name 

Values  by  Year 

1983 

0 

1984 

1985 

1986 

GZ  ( 1 ) 

311.0 

396.5 

5.8.5 

583.0 

GZ  ( 2 ) 

85.4 

108.5 

130.4 

101.4 

AKD 

0.154 

0.167 

0.168 

0.164 

U 

0.068 

0.068 

0.068 

0.068 

LB 

1131.1 

1153.6 

1145.5 

1165.1 

DG 

261 . 1 

301.1 

381.5 

346.5 
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Table  9.3  Extreme  Values  Used  in  the  Structured  Sensitivity  Analysis 


Variable 

Name 

Values  by  Year 

1983 

1984 

1985 

1986 

GZ(1) 

MIN 

290.2 

374.5 

471.4 

513.0 

MAX 

331.7 

418.6 

565.7 

653.0 

GZ  ( 2 ) 

MIN 

82.0 

101.2 

121.7 

94.6 

MAX 

88.8 

115.7 

139.0 

108.2 

AKD 

MIN 

0.150 

0.164 

0.165 

0.161 

MAX 

0.156 

0.170 

0.171 

0.167 

U 

MIN 

0.065 

0.065 

0.065 

0.065 

MAX 

0.071 

0.071 

0.071 

0.071 

LB 

MIN 

1110.3 

1132.4 

1124.5 

1143.7 

MAX 

1151.8 

1174.7 

1166.5 

1186.5 

DG 

MIN 

226.2 

284.4 

339.0 

308.0 

MAX 

295.8 

317.9 

423.8 

385.0 

82 


9.2  IMPACTS  OF  THE  MOBILIZATION  SCENARIO  ON  THE  NATIONAL  ECONOMY 


9.2.1  Analysis  of  the  Mobilization  Baseline 

The  volume  of  economic  output  increases  as  a result  of  mobilization.  Demand 
increases  due  to  high  government  purchases,  while  the  effective  volume  of 
available  resources  also  rises,  as  a result  of  the  increase  in  capital  and  labor 
services.  Thus,  demand  is  pulling  production  higher  while  resources  and  capacity 
permit  an  accompanying  increase  in  production. 


Total  Production 

Real  GNP  changes  in  1983  are  estimated  to  be: 

1.  Base  case  real  GNP  $3277.7  (10^); 

2.  Real  GNP  with  mobilization  $3634.4; 

3.  Increase  in  real  GNP  $356.7;  and 

4.  Percentage  increase  in  real  GNP  10.9 

The  higher  level  of  real  GNP  is  continued  from  1984  through  1986  where  the 
relative  increases  are  11.5,  11.3,  and  10.8  percent. 

Thus,  the  mobilization  stimulates  economic  activity.  Production  responds; 
the  entire  level  of  economic  activity  is  increased.  Real  GNP  increases  by  an 
average  of  11.1  percent  compared  to  the  base  case  economic  situation.  How  is 
this  increase  in  production  sustained?  At  an  aggregate  level,  the  increase  can 
be  separated  into  three  components: 

1.  Additional  input  of  labor  services; 

2.  Additional  input  of  capital  services;  and 

3.  Increases  in  productivity. 

Use  of  Output 

This  gain  in  production  is  used  in  the  form  of  higher  levels  of  final 
demand.  But,  which  types  of  spending  benefit  from  the  higher  level  of  economic 
activity?  Clearly,  government  purchases  will  increase  substantially,  since  the 
rise  in  defense  spending  is  the  initiating  factor  in  all  these  adjustments.  But, 
do  consumption  and  investment  share  in  the  greater  level  of  production?  The 
average  increases  in  final  demand  during  the  period  of  mobilization  are: 


1 . 

Personal  consumption  expenditures 

6.2% 

2. 

Gross  private  domestic  investment 

13.1% 

3. 

Government  purchaes  38.0% 

4 . 

Exports  -50.0% 

5. 

Imports  -20.0% 
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Consumption  and  investment  rise  and  government  purchases  rise 
Thus,  the  gains  from  the  shift  to  mobilization  do  not  accrue  to  bu 
Instead,  the  gains  go  to  the  government.  Mobilization  generates  a 
economic  structure,  away  from  business  and  investment  and  consumpt 
government  activity. 


most  of  all. 
si nesses . 

shift  in 
ion  and  towards 


Economic  Growth 


The  initial  effect  of  the  mobilization  is  to  raise  the  level  of  economic 
activity.  Real  GNP  in  1983  is  some  10.9  percent  higher  than  in  the  base  case. 
But,  does  this  mean  that  the  mobilization  generates  more  rapid  economic  growth? 

In  fact,  the  mobilization  does  not  increase  economic  growth.  Certainly,  the 
growth  in  real  GNP  from  1983  is  accelerated.  But,  this  corresponds  to  a one-time 
jump  to  a higher  level  of  economic  activity.  Once  production  is  at  this  new, 
higher  level,  growth  rates  revert  to  their  former  pattern.  Thus,  mobilization 
generates  a single  shift  to  more  production,  as  both  final  demand  and  available 
resources  are  boosted.  Mobilization  does  not  generate  a continuing  gain  in 
achievable  rates  of  economic  growth. 

This  is  an  important  point,  for  it  means  that  the  early  gains  under 
mobilization  will  not  be  repeated.  A higher  level  of  production  is  sustainable, 
at  least  for  some  time,  but  large  continuing  gains  are  not  realistic. 

Prices 


The  mobilization  does  affect  the  level  of  prices.  It  might  be  expected  that 
inflation  will  accelerate  (i.e.,  that  the  rate  of  increase  in  the  general  level 
of  prices  will  rise).  This  is  exactly  what  happens.  The  GNP  price  deflator 
shows  an  acceleration,  over  the  GNP  price  deflator  for  the  base  case,  of  2.3 
percent  in  1983,  4.5  percent  in  1984,  9.7  percent  in  1985  and  13.8  percent 
in  1986.  The  estimated  inflationary  impact  is  not  immediate.  It  is  not  until 
the  third  year  of  the  mobilization  that  the  inflationary  effects  become 
striking. 


Industry  Effects 


Different  industries  are  affected 
These  impacts  reflect  several  forces, 
change  in  input  patterns  accompanying  a 


to  different  degrees 
The  rise  in  defense 
ny  restructuring  in 


by  the  mobiliza 
purchases;  and  a 
relative  prices. 


t 


ion. 


Overall  production  rises,  on  the  average,  by  10.1  percent  compared  to  the 
base-case  conditions.  The  percentage  change  in  individual  industries  ranges  from 
a decline  of  11.5  percent  for  metal  mining  to  a rise  of  32.1  percent  for 
transportation  equipment  and  ordnance. 


Several  industries  have  increases  in  output  volumes.  These  involve  crude 
petroleum  and  gas,  petroleum  refining,  gas  utilities,  electric  utilities, 
construction,  services,  and,  with  the  largest  gain,  military  transport  equipment 
and  ordnance.  Some  of  these  are  expected  as  they  are  the  materials  directly 
purchased  for  defense  purposes.  Others,  however,  reflect  the  indirect  effects, 
as  requirements  work  their  way  through  the  industrial  system.  For  example, 
military  transportation  equipment  and  ordnance  and  petroleum  are  directly 
related  to  defense  purchases,  whereas  most  of  the  remaining  industries  are 
indirectly  related. 
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In  contrast,  some  industries  are  estimated  to  experience  a reduction  in 
volume.  Even  though  overall  economic  activity  is  higher,  these  industries  are 
less  needed  than  in  the  base  case. 

Information  on  specific  industries  can  be  used  in  several  ways.  As  a 
projection,  it  estimates  what  would  happen  under  mobilization  conditions.  As  a 
prescription,  this  indicates  which  industries  are  most  important  to  the 
mobilization  effort.  Industries  with  a greater  than  average  increase  in  output 
are  of  greater  than  average  importance.  Some  of  these  are  obvious,  given  their 
direct  role  in  defense  purchases.  Others,  however,  are  less  obvious  and  the 
ability  to  reveal  such  industries  is  an  important  benefit  from  an  analysis  such 
as  this.  By  indicating  the  relative  importance  of  each  industry,  this  analysis 
can  help  mobilization  planning  if  policies  were  to  be  implemented  to  affect  the 
allocation  of  resources  (i.e.,  the  essence  of  such  allocations  is  to 
differentiate  higher  from  lower  priority  claimants  on  available  resources). 

A related  application  of  this  analysis  involves  the  identification  of 
bottlenecks.  The  increases  in  industry  outputs  are  the  increases  required  to 
sustain  the  mobilization.  These  increases  might  be  compared  with  the  expansion 
capability  of  an  industry  based  on  detailed  knowledge  of  the  industry  and  of 
constraints  specific  to  the  industry.  If  the  required  expansion  exceeds  the 
maximum  likely  expansion,  or  surge  capacity,  then  the  analyst  has  identified  a 
potential  bottleneck.  Contingency  plans  can  then  be  developed  to  ease  such 
bottlenecks  in  the  event  of  a mobilization. 

9.2.2  Results  of  the  Structured  Sensitivity  Analysis 

The  general  trends  described  in  the  previous  subsection  were  reinforced  in 
the  structured  sensitivity  analysis.  Consequently,  the  focus  here  is  on 
documenting  how  perturbations  about  the  values  of  the  six  key  input  variables 
given  in  Table  9.2  translated  into  changes  in  five  major  output  variables. 

These  output  variables  and  their  names  are: 

(1)  Real  Gross  National  Product,  GNPQ; 

(2)  Real  Personal  Consumption  Expenditures;  CNIAQ; 

(3)  Real  Gross  Private  Domestic  Investment,  INIAQ; 

(4)  Real  Private  Domestic  Capital  Services,  KD;  and 

(5)  Quantity  of  Energy  Delivered,  QED. 

The  first  three  variables  are  associated  with  the  National  Income  and 
Product  Accounts  (NIPA);  they  are  referred  to  in  the  text,  tables  and  figures 
which  follow  as  real  GNP , real  PCE  and  real  investment,  respectively.  The  fourth 
variable,  KD,  registers  the  joint  effects  of  variations  in  the  efficiency  with 
which  capital  is  employed,  AKD,  and  capital  accumulation  over  the  mobilization 
period.  It  is  important  to  note  that  capital  accumulation  incorporates  the 
effects  of  investment  and  another  key  input  variable  in  Table  9.2,  U,  the  rate 
of  depreciation  of  capital.  For  each  year  in  the  mobilization,  KD  is  equal  to 
the  product  of  AKD  and  KL,  the  total  real  private  domestic  capital  stock  at  the 
beginning  of  the  year.  The  fifth  variable  measures  the  amount  of  energy 
delivered  to  the  domestic  economy  in  quadrillion  BTU's  (Quads).  This  amount  is 
equal  to  the  sum  total  of  Quads  delivered  for  electricity,  coal,  natural  gas  and 
petroleum. 
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The  results  obtained  from  the  structured  sensitivity  analysis  on  each  of  the 
five  major  output  variables  are  described  through  reference  to  a series  of  tables 
and  figures.  These  tables  and  figures  summarize  three  basic  types  of  information 
for  each  variable: 

(1)  The  equilibrium  levels  experienced  as  a result  of  mobilization; 

(2)  The  percentage  change  (increase  or  decrease)  over  the  base-case 
simulation  as  a result  of  mobilization;  and 

(3)  The  annual  percentage  change  in  the  equilibrium  levels  during  the  period 
of  the  mobilization. 

These  three  types  of  information  are  hereafter  referred  to  as  mobilization 
values,  base-case  deviations,  and  annual  rates  of  change,  respectively. 

Most  of  the  figures  presented  in  this  section  make  use  of  a graphical 
analysis  technique  known  as  the  box  plot.^-  Box  plots  provide  a convenient  means 
for  summarizing  graphically  an  entire  data  set.  Box  plots  are  essentially  a 
graphical  analysis  of  variance.  One  advantage  of  box  plots  for  the  structured 
sensitivity  analysis  is  that  they  permit  multiple  years  worth  of  data  to  be 
compared  in  a single  diagram. 

The  box  plot  consists  of  a box  with  its  "hinges"  drawn  at  the  25th  and  75th 
percentiles,  respectively.  The  width  of  the  box  is  proportional  to  the  number  of 
observations.  The  median  (i.e.,  the  50th  percentile)  is  marked  within  the  box. 
The  extreme  values  are  marked  and  connected  to  the  hinges  of  the  box  with 
"whiskers."  The  box  plot  thus  graphically  summarizes  both  the  central  tendency 
of  the  data  as  well  as  its  dispersion.  Non-overlapping  boxes  on  the  same  plot 
are  a strong  indication  that  the  two  sets  of  data  are  significantly  different. 

All  Box  plots  presented  in  this  section  were  generated  through  application  of  the 
statistical  analysis  package  DATAPLOT.^ 

Growth  in  Total  Production 

As  was  shown  earlier,  the  output  of  the  economy  increases  during  the 
mobilization.  Figure  9.1  provides  a graphical  summary  of  the  economic  impacts  of 
mobilization  for  three  sets  of  values  for  real  GNP  between  1982  and  1986.  The 
first  year,  1982,  is  prior  to  the  mobilization  period  and  is  included  as  a 
reference  point.  The  first  set  of  values  is  for  the  base-case  simulation;  these 
values  are  traced  out  by  a solid  line.  These  values  provide  a basis  for 
comparing  the  output  of  the  peace-time  economy  to  outputs  anticipated  during  the 
mobilization.  The  second  set  of  values  is  for  the  mobilization  baseline;  these 
values  are  traced  out  by  a dotted  line.  The  third  set  of  values  is  for  the 
mobilization  maxima  (i.e.,  the  set  of  highest  values  for  each  year  across  all 
simulations);  these  values  are  traced  out  with  triangles  and  dashed  lines. 


^ JT  w!  Tukey,  Exploratory  Data  Analysis,  New  York:  Addison-Wesley , 1977. 

2 J.  J.  Filliben,  DATAPLQT:  Introduction  and  Overview,  Gaithersburg,  MD: 
National  Bureau  of  Standards,  Special  Publication  667,  1984. 
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Figure  9.1  Values  of  Real  GNP  for  the  Base-Case  Simulation,  the  Mobilization 

Baseline  and  the  Mobilization  Maxima 


YEAR  OF  MOBILIZATION 

BASE  CASE  - SOLID  LINE  MOBILIZATION  = DOTTED  LINE 
MOBILIZATION  MAXIMA  * TRIANGLES  ST  DASHED  LINE 
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Figure  9.2  Box  Plots  of  Mobilization  Values  for  Real  GNP 


YEAR  OF  MOBILIZATION 
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Through  reference  to  Figure  9.1,  it  can  be  seen  that  real  GNP  increased  from 
just  under  $3200  billion  in  1982  to  nearly  $4100  billion  in  1986.  The  figure 
documents  the  rapid  increases  in  real  GNP  in  1983  and  1984.  In  1985  and  1986, 
the  economy  gives  indications  of  a return  to  its  long-run  growth  trend,  since  the 
traces  associated  with  the  mobilization  parallel  that  of  the  base-case  scenario. 
The  figure  also  shows  that  mobilization  has  resulted  in  a net  increase  of 
approximately  $400  billion  in  the  level  of  real  GNP  over  the  base-case  simulation 
for  1984  through  1986. 

Figure  9.2  is  a sequence  of  box  plots  showing  the  range  of  the  mobilization 
values  for  real  GNP.  The  figure  demonstrates  that  significant  increases  in  the 
output  of  the  economy  are  experienced  in  each  year  of  the  mobilization.  A 
tabular  summary  of  the  data  in  Figure  9.2  is  given  in  Part  A of  Table  9.4.  From 
Part  A of  Table  9.4,  it  can  be  seen  that  the  range  (i.e.,  the  difference  between 
the  maximum  and  minimum  values)  increases  from  approximately  $55  billion  in  1983 
to  nearly  $80  billion  in  1986.  Although  the  standard  deviation  of  the  simulated 
values  increases  in  each  year,  it  averages  about  0.5  percent  of  the  mean. 
Consequently,  the  increases  in  real  GNP  noted  earlier  are  statistically 
significant . 

A quick  comparison  between  Figures  9.1  and  9.3  documents  the  magnitude  of 
the  change  in  the  output  of  the  economy  due  to  mobilization.  Figure  9.3 
indicates  that  the  war-time  economy  averages  an  increase  of  approximately  11 
percent  over  the  base-case  simulation.  Note  that  the  base-case  deviations  reach 
a peak  in  1984  and  then  decline  slightly  through  1986.  This  observation  is 
consistent  with  the  statement  made  earlier  that  the  economy  returned  to  its 
long-run  growth  trend  once  the  initial  shocks  of  the  mobilization  worked  their 
way  through  the  model.  Part  B of  Table  9.4  provides  a summary  of  the  data  in 
Figure  9.3.  These  data  show  the  significance  of  the  initial  surges,  followed  by 
the  resumption  of  a trend. 

Comparisons  between  Figures  9.1,  9.2,  and  9.4  and  the  data  for  1983  in 
Figure  9.3  illustrate  how  the  output  of  the  economy  has  grown  over  the 
mobilization  period.  The  initial  surge  in  government  purchases  produces  a rapid 
increase  in  output.  Reference  to  Figure  9.4  shows  that  the  annual  rate  of  change 
in  real  GNP  is  nearly  15  percent  in  1983.1  pn  1934^  the  annual  rate  of  change 
has  slowed  to  approximately  7.5  percent.  By  1985,  real  GNP  has  returned  to 
growth  rates  consistent  with  a peace-time  economy.  Part  C of  Table  9.4  provides 
a summary  of  the  data  in  Figure  9.4.  These  data  show  how  the  initial  surge 
produces  both  a high  rate  of  change  and  a high  variability  in  that  rate.  By 
1984,  although  the  rate  of  change  is  still  high,  the  variability  has  been  reduced 
substantially.  The  variability  in  the  growth  rate  then  hovers  around  0.1  percent 
through  1986.  These  observations  lend  support  to  the  claim  made  earlier  that 
mobilization  produces  a one-time  jump  to  a higher  level  of  economic  activity 
rather  than  a continuing  gain  in  achievable  rates  of  economic  growth. 


1 The  annual  rate  of  change  for  1983  is  computed  using  the  1982  value  of  the 
base-case  simulation.  Consequently,  the  annual  rate  of  change  of  nearly 
15  percent  incorporates  an  underlying  growth  trend  plus  a deviation  from  the 
base-case  simulation.  It  is  for  this  reason  that  the  annual  rate  of  change 
for  1983  exceeds  the  base-case  deviation  for  1983. 
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Table  9.4  Summary  Statistics  for  Real  GNP 


PART  A:  MOBILIZATION  VALUES 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

3629.6 

3606.4 

3659 . 7 

15.97 

1984 

3893.5 

3862.4 

3932.0 

19.58 

1985 

3970.9 

3936.7 

4010.0 

20.96 

1986 

4069.1 

4035.0 

4113.1 

23.44 

PART  B:  BASE-CASE  DEVIATIONS 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

10.74 

10.03 

11.66 

0.487 

1984 

11.50 

10.61 

12.60 

0.561 

1985 

11.23 

10.27 

12.32 

0.587 

1986 

10.77 

9.84 

11.97 

0.638 

PART  C:  ANNUAL  RATES  OF  CHANGE 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

14.64 

13.91 

15.60 

0.505 

1984 

7.27 

7.10 

7.44 

0.107 

1585 

1.99 

1.90 

2.13 

0.078 

1986 

2.47 

2.25 

2.61 

0.100 
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Figure  9.3  Box  Plots  of  Base-Case  Deviations  for  Real  GNP 


12  - 


1 1 - 


10  - 


9 — 
82 


83  84  85  86 


YEAR  OF  MOBILIZATION 


91 


Figure  9.4  Box  Plots  of  Annual  Rates  of  Change  for  Real  GNP 


YEAR  OF  MOBILIZATION 
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Use  of  Output 


The  focus  of  this  portion  of  the  analysis  is  on  changes  in  real  PCE  and  real 
investment.  The  other  NIPA  components,  government  purchases,  exports  and 
imports,  follow  the  same  patterns  as  in  the  mobilization  baseline.  Exports  are 
reduced  by  50  percent  and  imports  by  20  percent  in  all  simulations.  The  levels 
of  government  purchases  were  selected  according  to  the  Latin  hypercube  sampling 
scheme  described  earlier.  The  average  increase  in  government  purchases  was  38 
percent  over  the  period  of  the  mobilization. 

The  values  of  real  PCE  experienced  during  the  mobilization  are  plotted  in 
Figure  9.5.  From  the  figure,  it  can  be  seen  that  the  mobilization  causes  real 
PCE  to  increase  substantially  in  1983.  The  increases  in  real  PCE  for  1984 
through  1986,  however,  are  lower  than  in  the  base-case  simulation. 

The  way  in  which  real  PCE  changes  during  mobilization  period  is  clearly 
illustrated  in  Figure  9.6.  For  example,  real  PCE  for  1984  is  significantly 
higher  than  in  1983.  However,  once  the  initial  surge  in  government  purchases  has 
worked  its  way  through  the  model,  real  PCE  remains  nearly  constant.  These 
observations  are  summarized  in  Part  A of  Table  9.5,  where  the  mean  value  of  real 
PCE  changes  by  less  than  $20  billion  between  1984  and  1986. 

Comparisons  between  Figures  9.5  and  9.7  show  that  the  mobilization  values  of 
real  PCE  are  trending  toward  those  of  the  base-case  simulation.  Furthermore,  the 
trend  seems  to  be  significant  during  the  latter  years  of  the  mobilization. 
Reference  to  Part  B of  Table  9.5  shows  that  the  average  base-case  deviation  is 
6.2  percent.  The  value  for  1983  is  9.6  percent  whereas  it  is  only  2.4  percent 
for  1986. 

Figure  9.8  provides  some  additional  insights  into  how  real  PCE  changes  over 
the  mobilization  period.  After  an  initial  surge,  the  annual  rate  of  change  in 
real  PCE  is  well  under  5 percent.  Part  C of  Table  9.5  shows  that  the  growth 
rate  for  real  PCE  is  slightly  negative  for  1985  and  is  essentially  flat  for  1986. 

Real  Investment 

The  mobilization  values  of  real  investment  follow  cyclical  rather  than  a 
secular  trend.  These  values  plotted  in  Figure  9.9,  reflect  the  underlying 
business  cycle.  Investment  increases  sharply  in  1983  and  1984,  turns  down  in 
1985  and  experiences  a rate  of  increase  approximately  equal  to  that  of  the  base 
case  simulation  in  1986. 

The  box  plots  of  the  mobilization  values  of  real  investment  in  Figure  9.10 
indicate  that  all  changes,  both  increases  and  decreases,  are  likely  to  be  highly 
significant.  These  observations  are  borne  out  through  reference  to  Part  A of 
Table  9.6. 
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Figure  9.5  Values  of  Real  PCE  for  the  Base-Case  Simulation,  the  Mobilization 
Baseline  and  the  Mobilization  Maxima 


YEAR  OF  MOBILIZATION 

BASE  CASE  « SOLID  LINE  MOBILIZATION  » DOTTED  LINE 
MOBILIZATION  MAXIMA  = TRIANGLES  & DASHED  LINE 
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Figure  9.6  Box  Plots  of  Mobilization  Values  for  Real  PCE 
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Table  9.5  Summary  Statistics  for  Real  PCE 


PART  A:  MOBILIZATION  VALUES 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

2351.6 

2327.7 

2383.6 

14.86 

1984 

2423.0 

2395.1 

2453.1 

16.82 

1985 

2422.8 

2393.0 

2470.8 

22.59 

1986 

2442.6 

2406.9 

2499.5 

28.32 

PART  B:  BASE-CASE  DEVIATIONS 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

9.58 

8.47 

11.08 

0.692 

1984 

8.18 

6.93 

9.52 

0.751 

1985 

4.75 

3.46 

6.82 

0.976 

1986 

2.38 

0.89 

4.77 

1.187 

PART  C:  ANNUAL  RATES  OF  CHANGE 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

14.67 

13.51 

16.23 

0.725 

1984 

3.04 

2.79 

3.47 

0.190 

1985 

-0.01 

-0.75 

0.73 

0.418 

1986 

0.81 

0.33 

1.38 

0.309 
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Figure  9.7  Box  Plots  of  Base-Case  Deviations  for  Real  PCE 


YEAR  OF  MOBILIZATION 
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Figure  9.8  Box  Plots  of  Annual  Rates  of  Change  for  Real  PCE 


YEAR  OF  MOBILIZATION 
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As  was  the  case  for  real  PCE , the  early  gains  in  investment  due  to 
mobilization  are  narrowing  by  1985.  Figure  9.11  differs  from  Figure  9.7  in  that 
base-case  deviations  for  real  investment  appear  to  stabilize  at  around  7 percent. 

Figure  9.12  illustrates  the  importance  of  the  business  cycle.  Due  to  the 
combined  effects  of  the  accelerator  principle-*-  and  the  potential  for  crowding  out 
due  to  deficit  financing,  the  rate  of  change  in  real  investment  is  quite 
volatile.  Part  C of  Table  9.6  shows  how  volatile  investment  spending  is  during 
the  period  of  mobilization.  After  increasing  by  almost  41  percent  in  1983,  the 
rate  of  change  in  real  investment  drops  to  slightly  under  21  percent  in  1984;  it 
then  turns  sharply  negative  to  -8  percent  in  1985.  Investment  spending  then 
increases  in  1986,  although  the  level  is  still  less  than  at  its  peak  in  1984. 

* 

Capital  Services 

The  wide  swings  in  investment  experienced  during  the  mobilization  lead  one 
to  conjecture  about  their  impact  on  capital  services.  Recall  that  capital 
services  are  a function  of  the  level  of  the  capital  stock  at  the  beginning  of 
each  year  and  the  "efficiency"  with  which  that  stock  can  be  utilized. 

Figure  9.13  illustrates  that,  even  though  investment  is  quite  volatile, 
capital  services  increase  steadily.  By  1985,  capital  services  appear  to  be 
changing  according  to  the  same  long-term  trend  that  underlies  the  base-case 
simulation. 

The  pattern  of  tapering  off  shown  in  Figure  9.13  is  more  pronounced  in 
Figure  9.14.  In  particular,  after  significant  increases  in  capital  services 
experienced  between  1983  and  1985,  the  change  is  quite  modest  in  1986.  Reference 
to  Part  A of  Table  9.7  demonstrates  the  degree  of  overlap  for  capital  services 
in  1985  and  1986. 

From  Figure  9.13,  it  was  clear  that  capital  services  exceeded  those  under 
the  base-case  simulation.  It  is  now  important  to  determine  if  these  increases 
are  significant.  Reference  to  Figure  9.15  and  Part  B of  Table  9.7  indicates 
that  except  for  1983,  the  mobilization  level  of  capital  services  is  significantly 
higher  than  in  the  base-case  simulation.  Thus  investment  volatility  in  the 
war-time  economy  and  increased  shifts,  which  produce  increased  wear  and  tear  on 
the  entire  stock,  is  not  sufficient  to  reduce  the  "productive  flow"  of  services 
from  the  capital  stock. 


1 


The  accelerator  principle  asserts  that  net  investment  is  a function  of  the  rate 
of  change  in  final  output  rather  than  the  absolute  level  of  output. 
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Figure  9.9  Values  of  Real  Investment  for  the  Base-Case  Simulation,  the  Mobiliza- 
tion Baseline  and  the  Mobilization  Maxima 


YEAR  OF  MOBILIZATION 
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MOBILIZATION  MAXIMA  - TRIANGLES  Sf  DASHED  LINE 
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Figure  9.10  Box  Plots  of  Mobilization  Values  for  Real  Investment 


YEAR  OF  MOBILIZATION 
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Table  9.6  Summary  Statistics  for  Real  Investment 


PART  A:  MOBILIZATION  VALUES 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

629.8 

617.6 

645.1 

7.62 

1984 

761.4 

753.4 

775.0 

5.91 

1983 

700.1 

687.1 

720.1 

9.83 

1986 

739.1 

725.3 

759.9 

11.95 

PART  B:  BASE-CASE  DEVIATIONS 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

25.12 

22.68 

28.15 

1.515 

1984 

15.14 

13.92 

17.20 

0.894 

1985 

7.87 

5.88 

10.95 

1.514 

1986 

6.75 

4.75 

9.74 

1.726 

PART  C:  ANNUAL  RATES  OF  CHANGE 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

40.81 

38.07 

44.23 

1.705 

1984 

20.90 

20.02 

22.13 

0.740 

1985 

-8.06 

-8.79 

-7.09 

0.629 

1986 

5.58 

4.67 

6.39 

0.489 
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Figure  9.11  Box  Plots  of  Base-Case  Deviations  for  Real  Investment 
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Figure  9.12  Box  Plots  of  Annual  Rates  of  Change  for  Real  Investment 


YEAR  OF  MOBILIZATION 
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Figure  9.13  Values  of  Capital  Services  for  the  Base-Case  Simulation,  the  Mobili- 
zation Baseline  and  the  Mobilization  Maxima 


YEAR  OF  MOBILIZATION 
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Figure  9.14  Box  Plots  of  Mobilization  Values  for  Capital  Services 
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Table  9.7  Summary  Statistics  for  Capital  Services 


PART  A:  MOBILIZATION  VALUES 


YEAR 

MEAN 

MINIM  UM 

MAXIM  UM 

STD.  DEV. 

1983 

651.9 

642.3 

659.3 

5.41 

1984 

737.0 

727.2 

746.9 

5.64 

1985 

777.4 

766.1 

788.1 

6.47 

1986 

785.4 

770.3 

797.6 

8.09 

PART  B:-  BASE-CASE  DEVIATIONS 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

0.95 

LO 

. 

0 

1 

2.09 

0.838 

1984 

2.93 

1.56 

4.31 

0.788 

1985 

4.39 

2.87 

5.83 

0.869 

1986 

4.41 

2.39 

6.03 

0.108 

PART  C:  ANNUAL  RATES  OF  CHANGE 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

9.79 

8.16 

11.03 

0.911 

1984 

13.05 

12.49 

13.51 

0.389 

1985 

5.48 

5.17 

5.93 

0.191 

1986 

1.04 

0.24 

1.33 

0.344 
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Figure  9.15  Box  Plots  of  Base-Case  Deviations  for  Capital  Services 
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Figure  9.16  Box  Plots  of  Annual  Rates  of  Change  for  Capital  Services 


YEAR  OF  MOBILIZATION 


109 


Since  capital  services  are  based  on  the  quantity  of  capital  stock  available 
at  the  beginning  of  the  year,  the  largest  annual  rate  of  change  is  experienced  in 
1984.-*-  The  rate  of  change  in  capital  services  then  drops  markedly  due  in  part  to 
investment  volatility  and  the  high  rates  of  depreciation  included  to  reflect  wear 
and  tear  on  the  capital  stock.  Part  C of  Table  9.7  provides  a tabular  summary 
of  the  data  shown  in  the  box  plots. 

Quantity  of  Energy  Delivered 

Mobilization  results  in  a significant  increase  in  the  demand  for  energy. 

The  results  of  the  structured  sensitivity  analysis  indicated  major  increases  for 
each  type  of  energy  (coal,  electricity,  natural  gas,  petroleum)  treated  by  the 
model.  Consequently,  it  was  decided  to  focus  on  a summary  measure,  the  sum  total 
quantity  of  energy  delivered,  total  energy. 

Figure  9.17  illustrates  how  the  quantity  of  total  energy  increases  from  just 
under  50  Quads  to  in  excess  of  70  Quads.  After  the  initial  surge,  total  energy 
delivered  increases  only  slightly. 

Due  to  the  reduced  range  on  the  vertical  axis  in  Figure  9.18,  it  is  easier 
to  see  the  trend  in  the  quantity  of  energy  delivered.  As  was  the  case  for  real 
GNP,  total  energy  delivered  increases  sharply  between  1983  and  1984  and  then 
slackens.  This  trend  is  summarized  in  tabular  form  in  Part  A Table  9.8. 

The  base-case  deviations  for  total  energy,  shown  in  Figure  9.19,  follow  a 
similar  trend.  Part  B of  Table  9.8  indicates  that  the  trend  of  increasing 
quantities  delivered  in  the  first  three  years  is  likely  to  be  significant  while 
that  in  the  final  year  only  borders  on  significance. 

The  annual  rate  of  change  in  total  energy  delivered,  shown  in  Figure  9.20, 
follows  the  trend  seen  in  Figure  9.17.  After  an  initial  surge,  where  the  rate  of 
increase  approached  50  percent,  the  annual  rate  of  increase  settles  into  a tight 
band  which  is  less  than  5 percent.  Part  C of  Table  9.8  documents  the  more  than 
10-fold  decrease  in  the  annual  rate  of  change  between  1983  and  1984  through  1986. 
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Recall  that  the  largest  rate  of  change  for  investment  was  in  1983. 
Consequently,  capital  put  in  place  through  investment  in  1983  is  included  in 
the  quantity  of  capital  stock  available  at  the  beginning  of  1984. 


Table  9.8  Summary  Statistics  for  Total  Energy 


PART  A:  MOBILIZATION  VALUES 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

71.89 

71.62 

72.18 

0.183 

1984 

75.00 

74.65 

75.39 

0.223 

1985 

75.57 

75.16 

75.94 

0.258 

1986 

76.90 

76.48 

77.33 

0.312 

PART  B:  BASE-CASE  DEVIATIONS 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

42.46 

41.93 

43.02 

0.363 

1984 

44.13 

43.47 

44.88 

0.428 

1985 

46.62 

45.83 

47.33 

0.500 

1986 

47.38 

46.57 

48.21 

0.600 

PART  C:  ANNUAL  RATES  OF  CHANGE 


YEAR 

MEAN 

MINIMUM 

MAXIMUM 

STD.  DEV. 

1983 

46.57 

46.03 

47.15 

0.373 

1984 

4.32 

4.23 

4.45 

0.068 

1985 

0.76 

0.63 

0.88 

0.076 

1986 

1.77 

1 . 64 

1.92 

0.092 
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Figure  9.17  Values  of  Total  Energy  for  the  Base-Case  Simulation,  the  Mobilization 
Baseline  and  the  Mobilization  Maxima 


YEAR  OF  MOBILIZATION 

BASE  CASE  - SOLID  LINE  MOBILIZATION  = DOTTED  LINE 
MOBILIZATION  MAXIMA  = TRIANGLES  & DASHED  LINE 
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Figure  9.18  Box  Plots  of  Mobilization  Values  for  Total  Energy 
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Figure  9.19  Box  Plots  of  Base-Case  Deviations  for  Total  Energy 
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Figure  9.20  Box  Plots  of  Annual  Rates  of  Change  for  Total  Energy 
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10  SUMMARY  OF  MAJOR  FINDINGS 


This  report  developed  a set  of  generic  guidelines  which  may  be  used  to 
evaluate  emergency  management  models  and  data  bases.  In  order  to  provide  both 
an  explicit  statement  of  the  evaluation  methodology  and  a step-by-step  procedure 
for  implementing  it,  the  report  was  divided  into  two  parts.  The  basic  problem 
was  stated  in  Chapter  1 where  a three-phased  procedure  was  outlined  which 
focused  on: 

(1)  obtaining  a clear  and  comprehensive  statement  of  user  requirements 
and  objectives  pertaining  to  the  application  of  the  model; 

(2)  generating  appropriate  information  about  the  model  design  and 
performance  pertaining  to  user  requirements;  and 

(3)  evaluating  model  attributes  and  properties  according  to  predetermined 
criteria  of  performance  required  by  the  user. 

The  "analytical  techniques"  associated  with  the  second  phase  of  the 
procedure  form  the  core  of  the  generic  model/data  base  evaluation  guidelines; 
they  are  described  in  Part  I of  the  report.  The  stages  of  descriptive  analysis, 
program  verification  and  analysis,  data  audit,  and  sensitivity  analysis  provide 
the  pool  of  information  which  drives  the  evaluation  process.  If  one  views  the 
guidelines  as  having  a front  end  where  test  results  are  evaluated,  then  the 
analytical  techniques  provide  the  critical  linkage. 


The  previous  point  may  be  bett 
determine  what  the  user  requirement 
scenario,  it  is  necessary  to  unders 
documentation  which  purports  to  des 
system.  These  topics  are  the  subje 
One  must  also  recognize  that  in  the 
of  the  evaluation,  it  is  necessary 
and  which  instruments  are  to  be  use 
data,  control  variables,  parameters 
evaluator  deeper  into  the  topics  di 
and  analysis. 


er  understood  by  noting  that  in  order  t 
s are  and  how  they  translate  into  a tes 
tand  the  purpose  of  the  model  and  revie 
cribe  the  model,  as  implemented  on  the 
ct  of  Chapter  2 and  Sections  3.1  and  4. 

information  development  and  generation 
to  state  which  questions  are  to  be  stud 
d to  address  these  questions  (e.g.,  spe 
).  Answers  to  these  questions  take  the 
scussed  in  Chapter  3,  program  verificat 
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The  data  audit,  Chapter  4,  serves  to  firm  up  the  test  scenario  by  studying 
the  model-data  interface  and  the  analysis  of  file  computerization  and 
accessibility.  Specialized  statistical  techniques  are  also  used  to  characterize 
the  variability  of  the  input  data.  This  step,  in  conjunction  with  a review  of 
the  documentation  for  the  data  base,  provides  guidance  in  designing  the 
sensitivity  analysis. 


A structured  sensitivity  analysis  is  the  culmination  of  the  earlier  stages 
in  the  evaluation  process.  The  specialized  techniques  for  performing  a 
sensitivity  analysis  discussed  in  Section  5.2  seek  to  develop  relationships 
between  input  uncertainties  and  the  variability  of  the  model's  output,  to  find 
areas  where  the  model's  output  changes  drastically  (e.g.,  due  to  instabilities), 
or  to  exploit  unique  capabilities  of  the  model.  A major  goal  in  performing  a 
sensitivity  analysis  is  to  generate  the  critical  pieces  of  information  for  those 
acceptance  tests  which  focus  on  quantitative  characteristics. 
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How  the  process  of  information  development  and  generation  is  incorporated 
into  a generic  model/data  base  evaluation  procedure  is  illustrated  in  Part  11  of 
the  report.  The  Dynamic  General  Equilibrium  Model  (DGEM)  is  used  as  a test 
vehicle  for  the  generic  evaluation  guidelines.  DGEM  was  selected  for  evaluation 
primarily  for  its  unique  approach  to  modeling  emergency  situations.  The 
availability  of  documentation  was  also  a major  factor.  We  wish  to  alert  the 
reader  to  the  importance  of  documentation  in  any  model  evaluation  effort  because 
it  is  the  principal  means  through  which  performance  against  user  requirements 
can  be  measured. 

The  analysis  of  the  U.S.  economy  during  a military  mobilization  is  used  to 
illustrate  how  the  evaluation  procedure  may  be  applied  to  DGEM . This  approach 
was  taken  for  two  reasons.  First,  a mobilization  represents  a major 
perturbation  about  the  "business  as  usual"  base-case  simulation.  Consequently, 
any  weaknesses  of  the  model  (e.g.,  instabilities,  implausible  results,  etc.) 
should  be  revealed  by  such  a perturbation.  Second,  a mobilization  requires  an 
explicit  treatment  of  the  joint  interactions  of  several  factors  (e.g., 
international  trade,  the  business  cycle,  available  capacity,  investment,  etc.). 

The  formulation  of  the  mobilization  scenario  used  in  the  evaluation  of  DGEM 
was  a two-stage  process.  In  the  first,  a mobilization  baseline  was  constructed 
which  made  explicit  the  perturbations  about  the  business  as  usual  values  in  the 
DGEM  data  base.  The  mobilization  baseline  also  provided  a point  of  reference 
for  how  the  joint  interactions  of  these  perturbations  affected  the  path  of  the 
national  economy.  In  the  second,  six  key  variables  were  varied  in  combination 
subject  to  an  experimental  design.  This  enabled  us  to  explore  in  detail  certain 
patterns  which  were  uncovered  in  the  mobilization  baseline. 

The  previous  discussion  and  material  presented  in  Chapter  9 demonstrates 
that  there  are  at  least  four  reasons  why  DGEM  can  be  a useful  tool  for  analyzing 
the  economic  impacts  of  a military  mobilization.  First  and  foremost,  DGEM  was 
designed  to  deal  with  the  demand  surges  and  resource  constraints  which  are 
expected  during  the  transition  from  a peace-time  to  a war-time  economy. 
Specifically,  DGEM  explicitly  addresses  factors  which  affect  both  demand  and 
supply  on  an  annual  basis  up  through  the  year  2000.  Second,  the  TUTOR  provides 
the  user  with  a set  of  step-by-step  instructions  for  incorporating  information 
on  Defense  and  other  government  expenditures  as  well  as  economic  and  technical 
considerations  which  specify  how  the  economy  mobilizes.  Third,  from  a review  of 
the  documentation,  it  became  clear  that  much  care  went  into  the  design, 
development  and  testing  of  the  model.  Furthermore,  the  model's  documentation  is 
quite  complete  and  should  be  sufficient  to:  (1)  set  it  up  on  the  host  system; 

(2)  execute  a base-case  simulation;  (3)  interpret  the  results  of  the  base-case 
simulation;  (4)  create,  run,  and  interpret  user-specified  simulations.  Finally, 
the  DGEM  source  code  has  been  made  available  to  a number  of  users,  enabling  them 
to  explore  the  major  linkages  within  the  model  as  well  as  trace  the  flow  of 
calculations  for  particular  variables  of  interest. 
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In  Chapter  6,  a set  of  four  user  requirements  were  postulated.  For 
ease  of  reference,  each  requirement  is  summarized  briefly;  a point-by-point 
review  is  then  given.  Requirement  1 focuses  on  model  usage  (i.e.,  the 
operational  model).  Requirement  2 addresses  the  clarity  and  completeness  of  the 
documentation  for  the  conceptual  and  mathematical  models.  Requirement  3 focuses 
on  the  relationships  between  the  National  Income  and  Product  Accounts  (NIPA)  and 
the  model’s  output.  Requirement  4 is  concerned  with  a variety  of 
economic-technical  attributes. 


In  the  discussion  which  follows,  material  is  presented  which  compares 
critically  the  information  generated  in  the  model  evaluation  process  against 
user  requirements.  This  comparison  enables  us  to  conclude  that  DGEM  is  both  an 
appropriate  and  useful  model  for  analyzing  the  types  of  problems  anticipated 
during  a military  mobilization. 

Requirement  1 

must  have  documentation  sufficient  to  enable  an  analyst  to: 
set  it  up  on  the  host  system; 
execute  a base-case  simulation; 

interpret  the  results  of  the  base-case  simulation;  and 
create,  run,  and  interpret  user-specified  simulations. 

Narrative 

Item  a:  The  documentation  for  DGEM  is  quite  complete  and  should  be  sufficient 

for  an  analyst  to  set  it  up  on  the  host  system. 

Item  b:  The  TUTOR  provides  a convenient  and  essentially  "fool  proof"  means  for 

specifying  and  executing  a base-case  simulation. 

Item  c:  Several  parts  of  the  DGEM  documentation  provide  detailed  descriptions 

of  the  base-case  simulation.  Furthermore,  these  descriptions  provide  valuable 
insights  into  the  DGEM  analysis  framework,  which  should  be  helpful  to  users  in 
specifying  their  own  simulations. 

Item  d:  The  DGEM  documentation  and  the  structure  of  the  TUTOR  are  sufficient  to 

enable  an  analyst  to  create,  run,  and  interpret  a user-specified  simulation. 

The  TUTOR,  through  its  nested  set  of  menus,  facilitates  the  process  of 
specifying  a simulation.  In  many  cases,  questions  posed  to  the  analyst  by  the 
TUTOR  provide  reference  to  the  values  of  key  variables  contained  in  the 
base-case  simulation.  Consequently,  the  analyst  may  specify  major  portions  of 
the  desired  simulation  via  a set  of  "perturbations"  about  the  base-case 
simulation. 


The  model 

(a) 

(b) 

(c) 

(d) 
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Requirement  2 


The  structure  of  the  model  should  be  clearly  documented, 
includes  information  on  the  model’s: 


This  requirement 


(a)  policy  context; 

(b)  objectives; 

(c)  exogenous  and  endogenous  variables; 

(d)  equation  structure;  and 

(e)  data  base(s). 

To  the  extent  feasible,  the  user  should  be  assured  that  the  forecast  values 
of  the  "base-case"  simulation  have  been  compared  to  historically  achieved 
values.  Any  major  discrepancies  should  be  documented  and,  if  possible, 
explained . 

Narra  tive 


Item  a:  DGEM  provides  the  potential  user  with  significant  latitude  for 

specifying  policy  options  and  for  evaluating  their  economic  merits.  The  TUTOR 
is  especially  helpful  in  defining  how  the  model  can  address  mobilization- 
oriented  policies. 


Item  b:  It  is  anticipated  that  model  users  will  exhibit  a wide  range  of 

objectives.  The  documentation  (especially  the  User’s  Guide)  and  the  TUTOR 
should  be  sufficient  for  defining  the  user's  analytical  objectives  and  for 
translating  them  into  a runstream  for  executing  the  model.  At  an  aggregate 
level,  the  TUTOR  alone  may  be  sufficient  to  specify  an  approach  for  evaluating 
the  economic  impacts  of  both  the  increased  defense  purchases  and  resource 
constraints  (e.g.,  supply  interruptions)  anticipated  under  a military 
mobilization.  Some  sectoral  information  is  also  available.  For  example,  for 
any  of  the  36  sectors  treated  within  the  DGEM  analysis  framework,  the  analyst 
can  constrain  the  inputs  to  or  outputs  from  that  sector.  Such  an  analytical 
approach  can  be  useful  in  identifying  bottlenecks  or  assigning  priorities  to 
certain  industries  for  a particular  set  of  inputs. 


Item  c:  Information  on  the  model's  exogenous  and  endogenous  variables  is 

presented  in  36  DGEM:  The  Dynamic  General  Equilibrium  Simulation  Model.  This 

information  has  been  expanded,  reflecting  the  code  changes  described  earlier  as 
well  as  the  incorporation  of  the  TUTOR,  and  is  recorded  in  Appendix  E. 


Item  d:  The  equation  structure  of  DGEM  is  well  documented.  The  report,  36DGEM : 

The  Dynamic  General  Equilibrium  Simulation  Model,  includes  explicit  mathematical 
descriptions  of:  (1)  submodels  of  producer  behavior,  one  for  each  of  the  36 

domestic  producing  sectors;  (2)  a model  of  consumer  behavior;  (3)  balance 
equations  covering  physical  flows  through  the  interindustry  system  equating 
demand  and  supply  quantities  of  each  good  or  service  transacted;  (4)  market 
balance  equations  equating  value  of  expenditure  and  receipts  for  each  good  or 
service  transacted;  (5)  financial  identities  aggregating  value  flows  into 
aggregate  income,  financial  and  economic  accounts;  and  (6)  government  and  rest 
of  the  world  accounts. 
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Item  e:  The  DGEM  database  consists  of  19,327  entries  for  each  of  43  years 

(i.e.,  1958  through  the  year  2000).  Of  the  19,327  entries  for  each  year  5,903 
are  endogenous  variables,  1,643  are  exogenous  variables,  and  11,781  are 
coefficients.  The  estimation  of  the  model  over  the  historical  period  (i.e., 

1958  to  1974)  used  a database  constructed  by  Jack  Faucett  and  Associates.  In 
Chapter  8,  three  sets  of  base-case  simulations  were  reviewed;  they  were:  (1) 

model  performance  over  the  historical  period  (i.e.,  simulations  between  1959  and 
1974);  (2)  model  performance  between  1975  and  1981,  when  published  data  on 
certain  key  variables  was  available;  and  (3)  the  base-case  projections  (i.e., 
simulations  between  1982  and  year  2000).  In  evaluating  DGEM’s  performance 
between  1975  and  1981,  one  must  recognize  that  while  many  variables  were  aligned 
to  observed  values,  many  were  not.  In  most  cases,  the  simulated  values  for 
these  compare  favorably  with  information  developed  from  published  sources.  For 
example,  comparisons  between  figures  published  by  the  Department  of  Labor  and 
values  from  the  database  for  1974  and  1980  showed  close  agreement  in  the 
relative  importance  of  each  industry  and  the  changing  nature  of  that  importance 
over  time. 

Requirement  3 


The  model  must  be  cross  referenced  to  the  National  Income  and  Product  Accounts 
(NIPA).  This  minimum  requirement  provides  a set  of  control  totals,  which  may  be 
used  to  disaggregate  the  model's  output  to  address  sector  specific  issues. 
Furthermore,  it  is  desirable  for  the  model  to  provide  information  on 
those  endogenous  and  exogenous  variables  likely  to  be  of  interest  to  decision 
makers.  This  information  may  be  useful  either  in  gaining  insights  regarding  the 
"path”  of  the  economy  to  a war-time  footing  or  in  preparing  customized  reports. 

Narrative 

Some  of  the  final  demand  and  expenditure  accounts  in  DGEM  are  based  on  an 
accounting  system  developed  by  Christensen  and  Jorgenson.  This  system  was 
specifically  designed  to  identify  and  handle  consistently  variables  involved  in 
economic  growth.  Because  of  this,  the  system  differs  slightly  from  the  National 
Income  and  Product  Accounts  (NIPA)  which  are  the  commonly  used  framework  for 
economic  accounting.  The  principal  difference  between  the  DGEM  and  NIPA 
accounting  systems  arises  in  the  treatment  of  household  sector  capital.  This 
capital  comprises  residential  property,  automobiles,  and  other  consumer 
durables.  Household  capital  is  explicitly  treated  as  capital  in  the  household 
sector  accounts.  Purchases  of  consumer  durables  are  included  in  investment,  not 
in  consumption,  and  the  imputed  value  of  services  from  this  capital  is 
calculated  and  included  in  consumption.  This  yields  a system  of  accounts  in 
which  investment  includes  all  additions  to  capital  stock  and  in  which 
consumption  comprises  only  current  goods  and  services.  The  NIPA  system  differs 
from  DGEM  in  that  it  includes  in  consumption  an  imputation  for  the  services  from 
owner-occupied  residential  structures  but  not  those  from  other  forms  of  consumer 
durables.  Consequently,  a bridge  is  used  to  map  between  the  two  accounting 
systems.  The  bridge  comprises  a set  of  stochastic  equations  and  identities  that 
permit  the  DGEM  final  demand  aggregates  to  be  expressed  in  NIPA  terms.  The 
stochastic  equations  map  model  variables  such  as  real  consumption,  real 
investment,  price  of  consumption,  and  price  of  investment  into  the  corresponding 
NIPA  variables.  The  identities  then  aggregate  expenditure  and  output  into  gross 
national  product  (GNP),  real  GNP , and  the  remaining  aggregates  in  the  standard 
national  income  accounts. 
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Requirement  4 


The  model  must  incorporate  the  following  economic-technical  attributes: 

(a)  the  increasing  importance  of  international  trade  over  the  past  decade; 

(b)  the  business  cycle  concept; 

(c)  the  changing  composition  of  the  Gross  National  Product  (GNP); 

(d)  the  concepts  of  investment,  capital  services,  depreciation,  and 
emergency  capacity; 

(e)  wage  and  price  variables; 

(f)  both  supply  (e.g.,  capital  and  labor  services  available)  and  demand 
(e.g.,  military  requirements)  concepts; 

(g)  an  explicit  treatment  of  both  fiscal  and  monetary  policy; 

(h)  dynamic  characteristics  whereby  production  and  consumption  decisions 
in  one  period  affect  the  economy  in  future  periods;  and 

(i)  bridges  to  or  from  key  variables  (e.g.,  from  the  NIPA  components  to 
specific  sectors  as  defined  by  the  Commerce  Department’s  4-digit 
Standardized  Industrial  Classification  (SIC)). 

Narrative 


Item  a:  DGEM  addresses  foreign  trade  both  as  a part  of  final  demand  (exports) 

and  as  a source  of  goods  and  services  (imports).  Via  the  TUTOR,  imports  can  be 
specified  on  an  industry-by-industry  basis;  tariffs  or  quantitative  controls  are 
available  for  limiting  imports.  Exports  can  be  altered  industry-by-industry  or 
the  overall  volume  of  exports  can  be  changed  by  keeping  the  composition 
unaltered.  Also,  there  are  considerations  for  the  balance  of  trade.  The 
balance  of  trade  can  be  unconstrained  or  it  can  be  subject  to  limits,  reflecting 
the  constraints  that  normally  exist. 


Item  b:  Between  1982  and  the  year  2000  economic  growth  is  projected  to  average 

just  over  3 percent  per  year.  This  represents  continued  growth  at  rates  less 
rapid  than  during  the  World  War  II  to  1973  period  but  more  rapid  than  during  the 
1973-1983  decade.  This  growth  is  slightly  more  rapid  in  the  1980’s  than  in  the 
1990 's.  Economic  growth  tends  to  be  uneven  or  cyclical  rather  than  regular. 
Hence,  there  are  cycles  around  the  overall  growth  trend.  A feature  of  many 
previous  cycles  has  been  the  influence  of  federal  policies,  with  expansionary 
policy  in  the  run  up  to  an  election  producing  strong  growth  then  the 
post-election  tightening  of  policy  leading  to  a recession.  The  correlation  is 
not  perfect  but  it  is  certainly  present.  Therefore,  an  election-related  cycle 
is  assumed  to  continue  over  the  DGEM  forecast  period. 


Item  c:  The  different  types  of  final  demand  (e.g.,  personal  consumption 

expenditures,  investment,  and  government  purchases)  display  very  different 
patterns  over  the  business  cycle.  Government  purchases  are  the  most  stable 
since  they  reflect  political  rather  than  market  forces.  Although  government 
purchases  may  show  uneven  growth  rates,  these  are  not  related  to  the  business 
cycle  and  generally  even  work  to  dampen  the  magnitude  of  the  cycle.  Personal 
consumption  is  intermediate.  Service  and  nondurables  purchases  are  very  stable, 
durable  purchases  are  cylical,  so  that  overall  consumption  volumes  show  only  a 
mild  cycle.  Private  investment  is  the  most  volatile  of  the  main  spending 
components.  In  recession  years,  investment  growth  is  zero  or  even  negative;  in 
the  recovery  years,  investment  volumes  grow  much  more  rapidly  than  the  rest  of 
the  economy.  The  composition  of  final  demand  between  personal  consumption 
expenditures,  private  investment  and  government  purchases  (i.e.,  how  the  output 
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of  the  economy  is  used)  is  projected  to  show  substantive  change  (see  Table  8.5 
where  the  composition  of  final  demand  in  both  current  and  constant  dollar  terms 
is  shown). 

Item  d:  Capital  markets  are  modeled  explicitly  by  DGEM . The  model  of  capital 

markets  within  DGEM  includes  private  saving,  private  investment,  government 
borrowing,  and  foreign  borrowing.  The  government  deficit  plays  an  important 
role  in  capital  markets  as  modeled  within  DGEM  because  it  represents  a "firm" 
demand  for  funds  in  those  markets.  In  cases  where  the  government  deficit  is 
financed,  the  government  claims  a larger  share  of  investible  funds  and  tends  to 
"crowd  out"  private  investment.  DGEM  provides  a model  of  investment  which 
incorporates  the  business  cycle  concept  with  the  potential  for  crowding  out  due 
to  deficit  financing.  Since  net  investment  is  related  to  the  rate  of  change  in 

final  output  rather  than  the  absolute  level  of  output,  the  rate  of  change  in 

real  investment  within  DGEM  is  quite  volatile.  Capital  services  are  a function 
of  the  level  of  the  capital  stock  at  the  beginning  of  each  year  and  the 
efficiency  with  which  that  stock  can  be  utilized.  Consequently,  even  though 
investment  is  quite  volatile,  capital  services  increase  steadily  over  the 
mobilization  period.  Both  the  efficiency  with  which  the  capital  stock  can  be 
utilized  and  the  rate  of  depreciation  of  that  stock  are  explicit  in  DGEM.  This 
is  essential  for  any  mobilization  application  because  it  enables  the  model  to 
capture  cases  where  the  existing  capital  stock  is  used  more  intensively  (e.g., 
going  to  multiple  shifts  or  increased  utilization  for  existing  multi-shift 
facilities)  and  consequently  is  subjected  to  greater  wear  and  tear.  Emergency 
capacity  can  be  modeled  indirectly  by  adjusting  the  efficiency  with  which  the 
overall  stock  of  capital  is  utilized.  This  approach  results  in  an  increased 

flow  of  capital  services  which  supports  the  higher  levels  of  output  required  in 

a mobilization  situation  but  ignores  the  net  additions  to  the  overall  stock  of 
capital.  In  this  important  aspect  DGEM  differs  from  ROCS,  another  emergency 
management  model  currently  used  by  FEMA  and  DoD  (see  Appendix  D) , which  uses 
explicit  sector-by-sector  estimates  of  emergency  capacity.  Two  other  concerns 
regarding  DGEM ' s modeling  of  capital  markets  should  also  be  noted. ^ First,  DGEM 
assumes  all  inputs  (including  durable  plant  and  equipment,  land  and  working 
capital)  are  instantaneously  adjustable  among  firms  and  sectors.  Consequently, 
there  is  no  notion  of  industry-specific  capital  plant  and  equipment.  This  has 
as  an  implication  that  the  production  estimates  from  DGEM  may  be  somewhat 
optimistic.  Second,  if  the  capital  service  price  formulation  used  in  DGEM  were 
estimated  using  the  theoretically  appropriate  ex  ante  cost  of  capital,  then  the 
new  parameter  and  elasticity  estimates  could  differ  considerably  from  those  now 
found  in  the  DGEM  data  base. 

Item  e:  Both  wage  and  price  variables  are  treated  in  the  DGEM  analysis 

framework.  Household  behavior  is  modeled  through  spending-saving  and 
work-leisure  decisions.  Prices  of  each  industry’s  output  are  endogenous, 
reflecting  input  prices,  productivity,  and  input  patterns.  The  price  setting 
procedure  of  DGEM  is  based  on  the  following  assumptions:  (1)  all  sectors  are 

characterized  by  constant  returns  to  scale;  (2)  there  is  perfect  competition  in 
the  input  and  output  markets;  and  (3)  all  inputs  are  instantaneously  adjustable 
among  firms  and  sectors.  Given  the  above  assumptions,  it  follows  that  economic 
profits  are  zero  at  each  point  in  time  for  each  sector  and  that  price  equals 
average  and  marginal  cost.  In  summary,  price  setting  takes  place  by  setting 


^ Berndt , E~!  r7, 


op  cit . , 


pp  11-6. 
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price  equal  to  average  cost  (which  in  turn  equals  marginal  cost).  Prices  are 
therefore  independent  of  the  level  of  demand  for  the  product.  Once  these 
prices  are  set,  absolute  demands  for  outputs  and  inputs  can  then  be  obtained. 

In  view  of  the  previously  stated  assumptions  and  a non-substitution  theorem  put 
forward  by  Samuelson,  all  industry  1-0  coefficients  are  independent  of  the 
structure  and  distribution  of  final  demand. 1 On  the  labor  side,  labor  supply  is 
the  outcome  of  consumer's  optimizations,  while  labor  demand  is  the  sum  of 
demands  from  optimizations  in  each  of  the  36  producing  sectors.  For  any  given 
real  wage  rate,  labor  supply  is  determined  endogenously.  The  number  of 
unemployed  workers  (determined  exogenously)  is  then  subtracted  from  the  labor 
supply  to  get  a "net"  labor  supply.  Real  wage  rates  are  then  adjusted  so  that 
the  leisure-work  choice  decision  minus  exogenous  unemployment  results  in  a net 
labor  supply  that  just  equals  total  labor  demand.  Since  unemployment  is  modeled 
exogenously,  the  wage  rates  in  DGEM  are  not  those  that  clear  labor  markets  but 
rather  those  that  clear  markets  given  exogenously  set  unemployment  rates.  One 
may  therefore  conclude  that  while  the  model  has  very  appealing  general 
equilibrium  features,  the  labor  market  is  classical  in  that  unemployment  cannot 
be  explained. 

Item  f:  The  DGEM  analysis  framework  is  based  on  the  application  of  econometric 

modeling  to  input-output  analysis.  Where  input-output  analysis  assumes  fixed 
input-output  coefficients  at  any  point  in  time  DGEM  provides  for  flexible 
input-output  coefficients  induced  by  price  variations  in  primary  inputs  which 
are  associated  with  economic  policies  or  anticipated  contingencies.  The 
complete  model  consists  of  an  interindustry  model  and  macroeconomic  model  that 
integrates  demand  and  supply  conditions  for  consumption,  investment,  capital  and 
labor.  The  macroeconomic  model  separates  economic  activity  into  four  types  of 
goods  and  services:  the  output  of  consumption  and  investment  goods;  and  the 

inputs  of  capital  and  labor.  The  demand  and  supply  of  each  commodity  is 
calculated  as  well  as  market  clearing  prices  and  quantities.  A household 
submodel  determines  the  demand  for  consumption  goods,  supply  of  savings,  and  the 
supply  of  labor.  A production  function  relates  the  output  of  consumption  and 
investment  goods  to  the  inputs  of  capital  and  labor  services,  given  the  level  of 
technical  efficiency.  The  two  other  components  of  GNP , government  expenditures 
and  exports,  are  exogenous  to  the  model.  The  interindustry  model  determines 
interindustry  transactions  for  36  sectors,  the  demand  for  primary  inputs  (labor, 
capital,  and  competitive  imports),  the  allocation  of  GNP  as  final  demand  among 
the  sectors,  all  total  sector  outputs.  The  technology  of  each  producing  sector 
is  represented  by  a price  possibility  frontier  that  determines  the  supply  price 
of  output  as  a function  of  the  prices  of  primary  and  intermediate  inputs  and  the 
level  of  technical  efficiency. 

Item  g : The  version  of  DGEM  tested  in  this  study  did  not  include  a monetary 

policy  model. ^ Via  the  TUTOR,  a wide  range  of  fiscal  policy  measures  which  are 
available  to  the  U.S.  government  for  use  during  or  after  a major  event,  such  as 
military  mobilization,  may  be  modeled.  Some  of  these  policies  involve  purchases 
of  goods  and  services;  some  policies  control  the  allocation  and  use  of  available 
resources,  in  particular  labor  and  capital;  some  policies  control  production 
activities,  the  level  of  operation  of  specific  industries;  and  some  policies 
affect  fiscal  conditions,  such  as  government  spending,  taxation,  and  deficits. 

Berndt,  e7  r7,  op  cit,  pp  11-6. 

o 

A newer  version  of  DGEM  is  currently  available  which  incorporates  both 
monetary  and  fiscal  policy. 
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These  types  of  policies  are  included  and  can  be  specified  by  the  analyst.  For 
example,  fiscal  policy  involves  government  revenues  and  expenditures,  and  their 
difference,  the  government  deficit.  Government  revenues,  primarily  taxes, 
reduce  the  level  of  private  disposable  income  and  spending.  Government 
purchases  are  themselves  an  element  of  spending  and  directly  add  to  private 
spending  in  demand  for  goods  and  services.  The  government  deficit  affects  the 
capital  markets,  since  the  deficit  is  financed  by  government  borrowing.  The 
larger  the  deficit,  the  more  private  capital  is  preempted  (i.e.,  crowded  out), 
reducing  the  availability  of  funds  for  private  spending,  and  showing  up 
particularly  in  its  effect  on  private  investment.  Each  of  these  policy  measures 
is  available  to  the  government  and  each  can  be  adjusted  by  the  analyst  in 
designing  the  analysis  of  the  event. 

Item  h:  DGEM  models  the  U.S.  economy  as  a sequence  of  annual  equilibria  that 

determine  demand,  supply,  and  the  relative  prices  for  all  commodities.  In  each 
period  the  supply  of  capital  is  fixed  initially  by  past  investments  and 
depreciation.  Variations  in  demand  for  capital  services  by  producing  sectors 
and  households  affect  the  price,  but  not  the  quantity  of  capital  services. 
However,  present  investment  plans  will  alter  the  future  level  of  the  capital 
stock.  Similarly,  in  each  period  the  available  labor  time  is  fixed  by  past 
demographic  developments,  but  variations  in  the  demand  for  labor  time  by  the 
producing  sectors  and  by  the  households  for  leisure  consumption  affect  the  price 
of  labor  and  the  allocation  of  labor  time  between  work  and  leisure.  In 
addition,  the  supply  of  saving  is  simultaneously  determined  in  the  household 
model  and  must  be  in  equilibrium  with  respect  to  investment  demand.  In  each  new 
period,  an  equilibrium  is  determined  for  all  factor  and  product  markets  by 
integrating  a macroeconomic  model  with  an  interindustry  model  (see  the 
discussion  under  Item  f). 

Item  i:  At  the  present  time,  DGEM  consists  of  36  producing  industries.  More 

precisely,  the  DGEM  analysis  framework  includes:  (1)  inputs  to  each  industry 

from  each  of  the  36  producing  industries  and  from  capital,  labor  and  competitive 
imports;  and  (2)  outputs  from  each  industry  to  each  of  the  36  producing 
industries  and  to  final  demand  — • personal  consumption,  investment,  government 
purchases  (specifically,  federal  defense,  federal  nondefense,  and  state  and 
local  government),  and  exports.  Bridge  tables,  either  within  DGEM  or  between 
the  TUTOR  and  DGEM,  are  of  crucial  importance  for  most  military  applications. 

The  reasoning  behind  the  previous  statement  is  that  bottlenecks  are  more  likely 
to  occur  as  the  sectoral  level  of  detail  is  increased.  Conversely,  the  higher 
the  level  of  aggregation,  the  less  likely  is  the  analyst  to  be  able  to  identify 
a potential  bottleneck.  Based  on  the  previous  discussion,  it  should  be  clear 
that  bottlenecks  within  DGEM ' s 36  sector  framework  can  be  identified,  analyzed 
and  to  a certain  extent  alleviated.  The  question  thus  becomes  one  of  sectoral 
detail.  The  discussion  in  Appendix  D,  where  a multimodel  approach  to  military 
mobilization  is  reviewed,  expands  upon  this  subject.  It  is  also  worth  noting 
that  the  sectoral  level  of  detail  of  DGEM  differs  significantly  from  two  other 
military  mobilization  models,  ROCS  (115  sectors)  and  DEIMS  (400  sectors),  as 
well  as  the  BEA  benchmark  input-output  table  (537  sectors).  This  lack  of 
sectoral  detail  is  probably  the  most  serious  deficiency  of  DGEM  with  regard  to 
military  mobilization  applications. 

Based  on  our  review  of  DGEM,  both  the  model’s  source  code  and  documentation  and 
the  TUTOR,  it  is  clear  that  the  addition  of  two  bridge  tables  would 
substantially  enhance  the  ability  of  DGEM  to  provide  a detailed  analytical 
framework  for  a wide  variety  of  military  mobilization  applications.  The  first 
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is  a "pre-processor"  which  incorporates  a bridge  into  the  TUTOR  between  the  55 
defense  procurement  categories  used  in  DEIMS  and  DGEM ' s 36  producing  sectors. 
Alternatively,  the  "pre-processor"  bridge  table  could  be  between  DoC's  defense 
procurement  categories  list  and  DGEM  ' s 36  producing  sectors.  The  discussion  in 
Appendix  D addresses  some  of  the  pros  and  cons  of  both  DoD's  and  DoC’s  bridge 
tables  between  their  respective  defense  procurement  categories  and  the  economy's 
producing  sectors.  The  second  is  a "post-processor"  which  translates  the  DGEM 
output  into  a form  which  is  consistent  with  DoC's  4-digit  Standard  Industrial 
Classification  (SIC)  level.  If  this  were  done,  the  output  of  DGEM  would  be 
consistent  with  the  BEA  benchmark  input-output  table.  We  note  that  the 
"pre-processor"  bridge  table  discussed  earlier  may  be  incorporated  in  a 
relatively  straight-forward  manner.  This  is  because  much  of  the  logic  has 
already  been  incorporated  into  the  TUTOR.  Recall  that  government  purchases  are 
separated  into  three  types  of  government  activity:  (1)  federal  defense;  (2) 

federal  nondefense;  and  (3)  state  and  local  government.  In  turn,  each  type  of 
government  activity  involves  purchases,  spread  over  a number  of  supplying 
industries.  For  federal  defense  purchases,  a rudimentary  bridge  table,  which 
distributes  defense  purchases  from  the  55  DoD  procurement  categories  to  the  36 
producing  sectors,  is  included  in  the  TUTOR.  The  analyst  can  therefore  control 
defense  spending  in  a variety  of  ways  within  this  structure.  Overall  levels  of 
defense  spending  can  be  altered,  leaving  the  composition  unchanged,  or  the 
composition  and  level  can  be  changed.  Specific  defense  procurement  categories 
can  be  altered,  year-by-year.  Federal  nondefense  purchases  and  state  and  local 
government  purchases  are  spread  over  the  36  producing  sectors.  Federal 
nondefense  purchases  can  also  be  altered  in  total  or  industry-by-industry. 

State  and  local  purchases  can  be  altered  in  total,  although  the  composition 
remains  the  same.  The  development  of  a "post-processor"  bridge  table  is 
somewhat  more  complex,  but,  in  principle,  most  of  the  data  exists  to  go  from  the 
NIPA  components  to  the  4-digit  SIC  level  of  detail. ^ 


^ The  discussion  in  Appendix  D outlines  one  way  in  which  this  activity  might  be 
accomplished . 
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APPENDIX  A:  CRITICAL  STAGES  IN  THE  DEVELOPMENT  OF  LARGE-SCALE 

COMPUTER-BASED  SYSTSEMS1 


The  purpose  of  this  appendix  is  to  describe  those  factors  to  consider  when 
managing  large-scale,  computer-based  model  development  activities.  The 
five-phased  approach  for  the  factors  presented  in  this  appendix  can  greatly 
improve  the  management  of  these  activities  and  make  large-scale,  computer-based 
models  more  responsive  to  user  needs.  The  more  responsive  these  models  are  to 
user  needs  the  more  effective  and  efficient  they  should  be  in  providing 
information  to  the  decision-making  process. 

The  following  factors  are  the  product  of  an  analysis  of  management 
weaknesses  which  are  inherent  in  modeling  development  problems.  These  factors 
also  suggest  procedures  which  are  intended  to  serve  as  a reference  document 
for  managing  the  development  of  large-scale,  computer-based  models. 

Recommendations  for  the  management  of  large-scale,  computer-based  model 
development  efforts  are  divided  into  five  separate  phases:  problem  definition; 

preliminary  design;  detailed  design;  evaluation;  and  maintenance.  In  each  phase 
we  describe  suggested  specific  duties  and  responsibilities  of  the  user  and  the 
developer.  Requirements  for  documentation  should  be  met  as  necessary  in  each  of 
the  five  phases. 

The  problem  definition  phase  is  primarily  intended  to  describe  those  tasks 
that  should  be  carried  out  by  the  user  before  agreeing  to  a model  design. 
Additionally,  the  phase  also  describes  design  considerations  and  when 
appropriate,  the  contractual  relationships  which  the  user  must  determine  before, 
deciding  whether  a preliminary  design  effort  should  begin.  During  this  phase, 
the  user  should  be  acquiring  a clear  definition  of  the  problem.  If  the 
information  being  acquired  during  this  phase  indicates  development  should  be 
stopped  before  the  end  of  the  phase,  the  user  should  be  prepared  to  terminate 
development  at  this  point.  To  determine  whether  to  continue  into  the  next 
phase,  the  user  should  consolidate  and  thoroughly  review  work  completed  during 
this  phase.  At  this  point  the  user  should  be  confident  that  his  needs  will  be 
met  by  the  model  without  making  changes  to  decisions  that  have  been  made.  Once 
the  decision  is  made  to  proceed  to  the  preliminary  design  phase,  a report  should 
be  prepared  which  provides  specific  guidelines  to  be  followed  in  the  remaining 
phases.  These  guidelines  represent  (contractual)  responsibilities  for  the 
development  of  the  model. 

The  preliminary  design  phase  includes  specification  of  the  information 
content,  general  programming  logic,  and  model  algorithms  necessary  to  develop  a 
useful  model.  Preliminary  design  of  the  model  should  be  conducted  by  the 
developer  with  information  input  and  direction  provided  by  the  user.  At 
completion  of  this  phase,  the  user  should  consolidate  and  thoroughly  review  work 
completed  on  these  factors.  At  this  point  the  user  should  be  confident  that  all 
of  the  specifications  necessary  to  develop  a useful  model  have  been  identified 


^Portions  of  this  appendix  are  excerpted  from:  Gass,  S.  I.  (editor), 

Utility  and  Use  of  Large-Scale  Mathematical  Models.  Gaithersburg,  MD: 
National  Bureau  of  Standards,  Special  Publication  534,  1976. 
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and  agreed  to,  and  will  not  need  to  be  changed  during  the  following  phases.  The 
user  must  then  determine  the  contract  or  work  agreement  process  to  be  followed. 
The  contract  or  work  agreement  should  include  specifications  for  the  model  and 
the  control,  documentation,  and  report  requirements  of  the  following  two  or 
three  phases,  as  applicable  (detailed  design,  evaluation,  and  maintenance). 

In  the  detailed  design  phase  the  developer  designs  the  model  logic  and 
prepares  detailed  programs.  Briefings  should  be  held  periodically  between  the 
the  developer  and  the  user.  One  purpose  of  these  briefings  is  to  provide  the 
user  with  the  knowledge  and  confidence  necessary  to  apply  the  model.  During 
this  phase,  the  user  should  continuously  reevaluate  the  design  being  implemented 
and,  if  necessary,  should  recommend  minor  changes  within  the  scope  of  the 
contract  or  work  agreement.  The  user  should  always  be  available  to  the 
developer  to  answer  questions  and  provide  needed  information.  Before  continuing 
into  the  next  phase,  the  user  should  consolidate  and  thoroughly  review  the  work 
completed  in  this  phase.  This  review  should  provide  the  software  manager  with 
sufficient  updated  information  to  determine  the  adequacy  and  responsiveness  of 
the  development  effort  at  this  point.  If  the  model  has  been  adequately 
developed  to  meet  the  user's  needs,  procedures  for  evaluation  of  the  model 
should  be  established,  documented  and  carried  out  in  the  next  phase. 

The  evaluation  phase  provides  for  the  final  check  of  the  model  as  a whole. 
This  operational  testing  supplements  the  evaluation  of  individual  model  programs 
conducted  during  earlier  phases.  The  user  has  ultimate  responsibility  for 
evaluating  the  adequacy  of  the  developed  model.  Actual  evaluation  of  the  model, 
however,  may  be  done  by  the  user,  both  user  and  developer,  and/or  an  independent 
third  party.  Evaluation  of  the  model  is  done  according  to  the  evaluation 
criteria  and  test  plan  established  in  the  problem  definition  phase.  Evaluation 
includes  the  determination  of  compliance  with  previously  established  agreements. 
At  the  completion  of  this  phase,  the  user  should  prepare  a report  based  on  the 
evaluation  work.  This  report  should  include  the  user’s  overall  satisfaction  or 
dissatisfaction  with  the  modeling  effort  and  the  final  model  design.  A decision 
should  then  be  made  as  to  whether  or  not  the  model  is  usable. 

In  the  maintenance  phase  the  Federal  agency  that  sponsored  model 
development  establishes  procedures  for  updating  the  model  and  for  obtaining  from 
the  users  their  comments  on  the  adequacy  of  the  model  and  whatever  changes  they 
made  to  the  model.  The  developer  should  be  available  for  assisting  the  user 
after  completion  of  model  development  in  accordance  with  the  agreement 
established  during  the  preliminary  design  phase.  Agency  management  should 
obtain  from  the  user  an  abstract  of  the  model  application  to  provide  information 
to  others.  Periodic  reports  should  be  prepared  showing  any  changes  made  to  the 
model  and  indicating  the  current  status  of  the  model.  When  the  model  can  no 
longer  meet  user  needs,  its  maintenance  should  be  stopped  and  the  status  should 
show  it  is  not  usable. 

In  summary,  the  factors  represent  some  suggested  procedures  for  model 
development.  They  are  intended  to  illustrate  at  least  one  method  of  enhancing 
the  user’s  perspective  of  modeling  and  reducing  the  chance  of  failure  during 
model  development.  They  are  presented  in  a form  that:  (1)  distinguishes  five 

separate  phases  of  model  development;  (2)  promotes  a more  thorough  early 
investigation  of  the  nature  of  the  problem  and  of  possible  solution  methods;  and 
(3)  provides  a method  of  controlling  the  commitment  to  a modeling  effort  during 
the  model’s  development  period. 
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APPENDIX  B:  SPECIALIZED  TECHNIQUES  FOR  PERFORMING  A DATA  AUDIT* 


As  discussed  in  Chapter  4,  the  statistical  techniques  used  in  a data  audit 
fall  into  two  major  classes:  (1)  exploratory  data  analysis  techniques;  and  (2) 

classical  statistical  techniques.  Although  the  approach  taken  for  each  class  of 
techniques  differs  significantly,  both  classes  focus  on  the  same  key  measures. 
These  measures  are:  (1)  central  tendency:  (2)  distributional  form;  and  (3) 

relational  properties.  In  the  text  which  follows,  emphasis  will  first  be 
placed  on  exploratory  data  analysis  techniques  <>  The  discussion  of  classical 
statistical  techniques  follows  immediately. 

An  important  summary  of  the  central  tendency  of  a data  set  is  a box  and 
whisker  plot.  This  graphical  display  highlights  five  important  characteristics 
of  the  data  set.  The  box,  usually  aligned  vertically,  encloses  the 
interquantile  range  of  the  data,  with  the  lower  line  identifying  the  25th 
percentile  and  the  upper  line  the  75th  percentile.  A line  sectioning  the  box 
displays  the  location  of  the  median  within  the  interquantile  range.  The 
whiskers  (line  segments)  at  either  end  may  extend  to  the  extreme  values  or,  for 
large  data  sets,  to  the  10th  or  90th  percentiles  or  5th  and  95th  percentiles. 

Box  plots  are  particularly  useful  for  comparing  the  measures  of  central  tendency 
for  two  or  more  data  sets  or  different  segments  of  the  same  data  set. 

Two  techniques  which  are  particularly  useful  for  measuring  the  central 
tendency  of  a data  set  as  well  as  providing  a summary  of  distributional  form  are 
histograms  and  stem  and  leaf  diagrams.  Both  techniques  sort  the  data  from 
smallest  to  largest  and  assign  them  to  a group.  Generally,  the  analyst  must 
select  the  interval  widths  or  the  number  of  groups  prior  to  the  analysis.  Once 
this  has  been  done,  a histogram  becomes  a graphical  representation  of  a 
frequency  distribution.  Histograms  typically  use  the  heights  of  bars  to  exhibit 
the  relative  frequency  of  occurrences  within  each  group  of  values  within  the 
data  set.  The  main  uses  of  a histogram  in  a data  audit  are:  (1)  to  condense  a 

set  of  data  for  easy  visual  comprehension  of  general  characteristics  such  as 
typical  values,  dispersion,  and  shape;  (2)  to  suggest  probability  models  or 
transformations  for  subsequent  analysis;  and  (3)  to  detect  unexpected  behavior 
or  unusual  values  in  the  data. 

An  important  variant  of  the  histogram  is  the  stem  and  leaf  diagram.  A stem 
and  leaf  diagram  has  the  appearance  of  a histogram  placed  on  its  side.  Each 
group  defines  a stem  and  each  piece  of  information  (item)  on  a stem  is  a leaf. 
The  stem  appears  on  the  left  side  of  a vertical  line  and  the  leaves  appear  on 
the  right  side  of  the  vertical  line.  The  leaves  may  be  labeled  so  that  the 
diagram  preserves  information  distribution  within  groups.  An  additional  feature 
of  these  diagrams  is  a summary  column  which  represents  the  cumulative  number  of 
observations  from  each  end  of  the  data  distribution  (termed  depths)  up  to  the 
group  containing  the  median.  The  major  advantages  of  stem  and  leaf  diagrams  are 
that  they  show:  (1)  clearer  separation  into  groups;  (2)  unsymmetric  trailing 

off;  (3)  unexpectedly  popular  and/or  unpopular  values;  (4)  where  the  data  are 
centered;  and  (5)  how  widely  the  data  are  spread. 


* Por tions  of  the  text  which  follows  are  abstracted  from:  Filliben,  J.  J. 
"Testing  Basic  Assumptions  in  the  Measurement  Process,"  in  Validation  of  the 
Measurement  Process,  New  York:  American  Chemical  Society,  ACS  Symposium  Series 

No.  63,  pp.  30-113,  1977. 
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Although  histograms  and  stem  and  leaf  diagrams  provide  some  insight  into 
the  underlying  distribution  of  the  data,  probability  plots  permit  specific 
hypotheses  about  the  distributional  properties  of  the  data  to  be  tested.  A 
probability  plot  is  a graphical  tool  for  assessing  the  goodness  of  fit  of  some 
hypothesized  distribution  to  an  observed  data  set.  A probability  plot  is  (in 
general)  simply  a plot  of  the  observed  ordered  (smallest  to  largest) 
observations , Y-^,  on  the  vertical  axis  versus  the  corresponding  typical  ordered 
observations , M-j_,  on  the  horizontal  axis  based  on  whatever  distribution  is  being 
hypothesized.  Thus,  for  example,  if  one  were  forming  a normal  probability  plot, 
the  following  n coordinate  plot  points  would  be  formed i (Yj,  M^)  (Y£, 

M2)»**(Yn,  Mn)  where  Y^  is  the  observed  smallest  data  point,  and  M]_  is  the 
theoretical  "expected"  value  of  the  smallest  data  point  from  a sample  of  size  n 
normally  distributed  points. 

Similarly,  Y2  would  be  the  second  smallest  observed  value  and  M2  would  be 
the  "expected  value"  of  the  second  smallest  observation  in  a sample  of  size 
n from  a normal  distribution.  Thus,  in  forming  a normal  probability  plot, 
the  vertical  axis  values  depend  only  on  the  observed  data,  while  the 
horizontal  axis  values  are  generated  independently  of  the  observed  data  and 
depend  only  on  the  theoretical  distribution  being  tested  or  hypothesized 
(normality  in  this  case)  and  also  the  value  of  the  sample  size  n.  A 
probability  plot  is  thus  in  simplest  terms  a plot  of  the  observed  versus 
the  theoretical  or  "expected." 

The  crux  of  the  probability  plot  is  that  the  i^  ordered  observation  in  a 
sample  of  size  n from  some  distribution  is  itself  a random  variable  which  has  a 
distribution  unto  itself.  This  distribution  of  the  ic^  ordered  observation  can 
be  theoretically  derived  and  summarized,  as  can  any  other  random  variable.  One 
can  then  pose  the  relevant  question  as  to  what  single  number  best  typifies  the 
distribution  associated  with  a given  ordered  observation  in  a sample  of  size  n. 
In  view  of  the  resistance  of  the  median  to  misbehavior  in  the  data,  the  Mp  on 
the  horizontal  axis  of  the  probability  plot  is  taken  to  be  the  median  of  the 
distribution  of  the  ith  ordered  observation  in  a sample  of  size  n from  whatever 
underlying  distribution  is  being  tested. 

It  is  noted  that  the  set  of  M^  as  a whole  will  change  from  one  hypothesized 
distribution  to  another  — and  therein  lies  the  distributional  sensitivity  of 
the  probability  plot  technique.  For  example,  if  the  hypothesized  distribution 
is  uniform,  then  a uniform  probability  plot  would  be  formed  and  the  M-^  will  be 
approximately  equi-spaced  to  reflect  the  flat  nature  of  the  uniform  probability 
density  function.  On  the  other  hand,  if  the  hypothesized  distribution  is 
normal,  then  the  M-j_  will  have  a rather  sparse  spacing  for  the  first  few  (M]_,  M2, 
M3...)  and  last  few  („..,  Mn_2»  Mn-i , Mn)  values  but  will  become  more  densely 
spaced  as  one  proceeds  toward  the  middle  of  the  set  (...,  Mn_i/2>  Mn/2> 
Mn+i/2>«*»)»  Such  behavior  for  the  M is  of  course  reflecting  the  bell-shape  of 
the  normal  probability  density  function. 

How  does  one  use  and  interpret  probability  plots?  In  light  of  the  above, 
it  is  seen  that  if  in  fact  the  observed  data  do  have  a distribution  that  the 
analyst  has  hypothesized,  then  (except  for  an  unimportant  location  and  scale 
factor  which  can  be  determined  after  the  fact)  the  Y^  and  M-j_  will  be 
near-identical  for  all  i,  that  is,  over  the  entire  set.  Consequently,  the  plot 
of  Y-l  versus  Mj_  will  be  near-linear . This  linearity  is  the  dominant  feature  to 
be  checked  for  in  any  probability  plot.  A linear  probability  plot  indicates 
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that  the  hypothesized  distribution,  DQ , gives  a good  distributional  fit  to  the 
observed  data  set.  This  combination  of  simplicity  of  use  along  with 
distributional  sensitivity  makes  the  probability  plot  an  extremely  powerful  tool 
for  data  analysis. 

Graphical  analysis  techniques  are  particularly  helpful  at  analyzing  relational 
properties.  Techniques  such  as  scatter  diagrams,  run  sequence  plots  and  lag-1 
autocorrelation  plots  are  important  tools  for  the  analyst.  A usual  first  step 
in  analyzing  the  relationship  between  two  variables  is  to  construct  a scatter 
diagram,  a collection  of  plotted  points  representing  the  pairs  (x^,  y-^), 
i — 1 , . . , n • 

Scatter  diagrams  are  also  a useful  precursor  to  regression  analyses.  The 
objective  of  regression  analyses  is  to  develop  a-  mathematical  relationship 
between  a measured  response  or  dependent  variable,  y,  and  two  or  more  predictor 
or  independent  variables,  xj  , X2,....Xp.  A usual  initial  step  is  to  plot  y 
versus  each  of  the  x variables  individually  and  to  plot  each  x versus  each  of 
the  other  x’s.  This  results  in  a total  of  p + p(p  - 1 ) / 2 plots.  Plots  of  y 
versus  x-j_,  enable  one  to  identify  the  x-^  variables  which  appear  to  have  large 
effects,  to  assess  the  form  of  a relationship  between  the  y and  x-^  variables, 
and  to  determine  whether  any  unusual  data  points  are  present.  Plots  of  Xj_  vs. 

Xj , i t j,  help  to  identify  strong  correlations  that  may  exist  among  the 
predictor  variables. 

The  run  sequence  plot  is  the  next  tool  for  analyzing  relational  properties. 

Although  the  collection  of  data  points  may  or  may  not  have  been  equispaced  in 

time,  the  ordering  of  the  data  in  time  (i.e.,  the  run  sequence)  is  usually 

well-defined.  In  cases  where  the  data  acquisition  rate  is  such  that  there  is  an 

equal  time-spacing  between  collected  data  points,  the  run  sequence  has  a natural 
analogue  to  a possibly  relevant  factor  (time).  In  other  cases,  when  the  data 
acquisition  rate  is  variable  or  random,  no  such  analogue  exists  --  yet  the  run 
sequence  "factor"  is  still  frequently  of  interest.  The  run  sequence  plot 
(defined  as  a plot  of  versus  i)  is  the  simplest  possible  data  plot  and  yet  is 
almost  invariably  informative.  The  run  sequence  plot  is  the  recommended  first 
step  in  assessing  whether  such  basic  assumptions  as  fixed  location,  fixed 
variation  and  the  implicit  corollary  assumptions  hold-up,  namely  that  the  data 
set  is  outlier-free.  With  regard  to  economic  data,  the  run  sequence  plot  may 
help  to  identify  seasonal  and/or  other  trends. 

The  lag-1  autocorrelation  plot  is  defined  as  a plot  of  Yj_  versus  Y^_i  over 
the  entire  data  set;  that  is,  the  following  n-1  points  are  plotted:  (Y2>  Y]_), 

(Y3,  Y2),  (Y4,  Y3),...(Yn,  Yn_i).  The  lag-1  autocorrelation  plot  is  sensitive 
to  randomness  assumptions  governing  the  data.  If  the  data  are  random,  then 
adjacent  observations  will  be  uncorrelated  and  the  plot  of  Yj_  versus  Y^_^  will 
appear  as  a data  cloud  with  no  apparent  structure.  However,  if  the  data  are 
not  random  and  if  adjacent  observations  do  have  some  autocorrelation  this 
structure  will  frequently  manifest  itself  in  the  autocorrelation  plot. 

Part  of  the  motivation  of  the  previous  discussion  was  to  demonstrate  how 
techniques  based  on  exploratory  data  analysis  complement  classical  statistical 
analysis  techniques.  In  the  discussion  which  follows,  we  shall  introduce  some 
of  the  most  important  classical  tehniques  with  particular  emphasis  being  placed 
on  how  these  techniques  can  draw  on  information  produced  through  an  exploratory 
examination  of  the  data. 
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As  a first  cut,  it  may  be  helpful  to  determine  the  minimum  and  maximum 
values  of  the  variables  within  the  data  set.  The  range  is  then  the  difference 
between  the  maximum  and  minimum  values.  It  is  also  helpful  to  determine  the 
mode,  or  most  popular  value,  as  well  as  the  median.  The  mean  is  the  most  common 
measure  of  central  tendency.  Often  referred  to  as  the  average,  it  is  merely  the 
sum  of  the  individual  values  for  each  case  divided  by  the  number  of  cases.  The 
variance  of  the  sample,  s^,  is  a measure  of  the  dispersion  of  the  data  about  the 
mean.  This  statistic  is  one  way  of  measuring  how  closely  the  individual  values 
of  the  variable  cluster  around  the  mean.  Mathematically,  it  is  the  average 
squared  deviation  from  the  mean.  Squaring  the  deviations  from  the  mean  takes 
into  account  all  differences  from  the  mean,  including  negative  differences,  and 
gives  additional  weight  to  extreme  cases.  The  standard  deviation  is  another 
measure  of  dispersion  about  the  mean;  it  is  the  square  root  of  the  variance. 

The  advantage  of  the  standard  deviation  is  that  it  has  a more  intuitive 
interpretation,  being  based  on  the  same  units  as  the  original  variable.  If  an 
infinite  number  of  equal-sized  samples  were  drawn  from  a given  population,  the 
mean  of  each  sample  would  be  an  estimate  of  the  true  population  mean,  but  not 
all  of  them  would  be  identical.  The  pattern  of  these  means  would  actually 
constitute  a normal  distribution  and  would  have  a standard  deviation.  The 
standard  deviation  of  this  distribution  is  the  standard  error.  Thus,  the 
standard  error  helps  to  determine  the  potential  degree  of  discrepancy  between 
the  sample  mean  and  the  usually  unknown  population  mean.  Skewness  is  a 
statistic  needed  to  determine  the  degree  to  which  a distribution  of  cases 
approximates  a normal  curve,  since  it  measures  deviations  from  symmetry.  The 
measure  of  skewness  is  sometimes  called  the  third  moment  and  will  take  on  a 
value  of  zero  when  the  distribution  is  completely  symmetric.  A positive  value 
indicates  that  the  cases  are  clustered  more  to  the  left  of  the  mean  with  most  of 
the  extreme  values  to  the  right.  A negative  value  indicates  clustering  to  the 
right.  Kurtosis  is  a measure  of  the  relative  peakedness  or  flatness  of  the 
curve  defined  by  the  distribution  of  cases.  A normal  distribution  will  have  a 
kurtosis  of  zero.  If  the  kurtosis  is  positive,  then  the  distribution  is  more 
peaked  (narrow)  than  would  be  true  for  a normal  distribution,  while  a negative 
value  means  that  it  is  flatter.  Kurtosis  is  sometimes  called  the  fourth 
moment . 

Partial  correlation  provides  the  analyst  with  a single  measure  of 
association  describing  the  relationship  between  two  variables  while  adjusting 
for  toe  effects  of  one  or  more  additional  variables.  In  essence,  partial 
correlation  enables  the  analyst  to  remove  the  effect  of  the  control  variable 
from  the  relationship  between  the  independent  and  dependent  variables  without 
physically  manipulating  the  raw  data.  In  partial  correlation  the  effect  of  the 
control  variable(s)  is  assumed  to  be  linear  throughout  its  range,  and  it  is  this 
linear  assumption  that  makes  partial  correlation  possible. 

Partial  correlation  can  be  used  in  a wide  variety  of  ways  to  aid  the 
researcher  in  understanding  and  clarifying  relationships  between  three  or  more 
variables.  When  properly  employed,  partial  correlation  becomes  an  excellent 
technique  for  uncovering  spurious  relationships,  locating  intervening  variables, 
and  can  even  be  used  to  help  the  researcher  make  certain  types  of  causal 
inferences.  Partial  correlation  can  be  a very  helpful  tool  for  enabling  the 
researcher  to  locate  spurious  relationships.  A spurious  correlation  is  defined 
in  a relationship  between  two  variables,  A and  B for  example,  in  which  Afs 
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correlation  with  B is  solely  the  result  of  the  fact  that  A varies  along  with 
some  other  variable,  C for  example,  which  is  indeed  the  true  predictor  of  B.  In 
this  case,  when  the  effects  of  C are  controlled,  held  constant,  etc.,  B no 
longer  varies  with  A. 


Another  important  feature  of  partial  correlation  lies  in  its  ability  to  aid 
the  analyst  in  a search  for  intervening  linking  variables.  While  there  is  no 
statistical  difference  between  the  computation  of  partials  employed  to  locate 
spurious  relationships  and  those  used  to  determine  intervening  variables,  the 
conceptual  issues  are  different  enough  to  merit  separate  treatment.  The  search 
for  intervening  variables  is  highly  related  to  the  issue  of  causality  insofar  as 
the  researcher  wishes  to  make  statements  of  the  sort:  A leads  to  B which  in 

turn  leads  to  C.  While  partial  correlation  can  be  of  great  assistance  in  such 
problems,  theoretical  considerations  become  much  more  important  in  these  types 
of  situations. 


Multiple  regression  is  a general  statistical  technique  through  which  one 
can  analyze  the  relationship  between  a dependent  variable  and  a set  of 
independent  or  predictor  variables.  Multiple  regression  may  be  viewed  either  as 
a descriptive  tool  by  which  the  linear  dependence  of  one  variable  on  others  is 
summarized  and  decomposed,  or  as  an  inferential  tool  by  which  the  relationships 
in  the  population  are  evaluated  from  the  examination  of  sample  data.  Although 
these  two  aspects  of  the  statistical  technique  are  closely  related,  it  is 
convenient  to  treat  each  separately,  at  least  on  a conceptual  level.  Since  the 
method  (as  a descriptive  tool  or  inferential  tool)  can  be  used  for  a variety  of 
related  purposes,  we  will  illustrate  only  a few  of  its  most  common  applications. 
The  most  important  uses  of  the  technique  as  a descriptive  tool  are:  (1)  to  find 

the  best  linear  prediction  equation  and  evaluate  its  prediction  accuracy;  (2)  to 
control  for  other  confounding  factors  in  order  to  evaluate  the  contribution  of  a 
specific  variable  or  set  of  variables;  and  (3)  to  find  structural  relations  and 
provide  explanations  for  seemingly  complex  multivariate  relationships. 


For  every  use  of  regression  as  a descriptive  tool,  there  is  usua 
corresponding  question  of  statistical  inference  — whether  one  can  ge 
the  results  of  the  sample  observation  to  the  universe.  The  problems 
statistical  inference  can  be  conveniently  grouped  into  two  general  ca 
estimation  and  hypothesis  testing.  The  purpose  of  estimation  is  to  f 
most  likely  population  parameters  from  the  examination  of  sample  obse 
The  main  focus  here  is  in  delineating  a particular  value  or  values  fo 
population.  The  analyst  may,  on  the  other  hand,  focus  on  evaluating 
hypotheses  about  the  population.  That  is,  instead  of  asking  what  val 
population  parameter  is  likely  to  have,  one  may  simply  test  the  null 
that  its  value  is  zero  against  the  alternative  hypothesis  that  its  va 
greater  or  less  than  zero. 
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As  mentioned  in  the  previous  subsection,  any  thorough  statistical  analysis 
should  focus  on  an  analysis  of  residuals.  If  a fitting  procedure,  such  as 
multiple  regression  is  used,  then  the  residuals  which  remain  are  assumed  to  be 
random.  The  runs  test  is  therefore  of  particular  importance  since  it  is 
specifically  used  to  test  for  randomness.  The  underlying  theory  behind  the  runs 
test  is  that  if  the  data  are  random  and  if  the  sample  size  is  known  (say  n=50), 
the  number  of  runs  up  of  length  1,  of  length  2,  etc. , may  be  considered  as 
random  variables  whose  expected  values  and  standard  deviations  can  be  calculated 


134 


from  theoretical  considerations  and  these  calculations  will  not  depend  on  the 
(unknown)  distribution  of  the  data  but  only  on  its  assumed  randomness.  Having 
computed  such  theoretical  values,  the  final  step  in  the  test  is  to  compute  from 
the  data  the  observed  number  of  runs  (up)  of  length  1,  of  length  2,  etc. , and 
then  determine  how  many  theoretical  standard  deviations  that  this  observed 
statistic  falls  from  the  theoretically  expected  value.  This  is  most  easily  done 
by  formation  of  the  standardized  variable: 

[N±  - E(Ni)]/SD(Ni) 

where  N-j_  is  the  observed  number  of  runs  (up)  of  length  i,  E(N-j_)  is  the 
theoretical  expected  number  of  runs  up  of  length  i and  SD(N-[)  is  the  theoretical 
standard  deviation  of  the  number  of  runs  of  length  i.  For  random  data,  one 
would  expect  values  of  say,  ±1,  ±2,  ±3,  i.e.,  the  observed  number  of  runs  of 
length  1 should  be  only  a few  (at  most)  standard  deviations  away  from  the 
theoretical  expected  value  for  the  number  of  runs  of  length  1.  For  nonrandom 
data,  the  deviations  from  the  expected  values  will,  of  course,  be  much  larger 
and  this  is  the  crux  of  the  runs  test. 

A great  deal  of  data  in  business,  economics  and  engineering  occur  in  the 
form  of  time  series,  where  observations  are  dependent  and  where  the  nature  of 
the  dependence  is  of  interest  in  itself.  The  body  of  techniques  available  for 
the  analysis  of  such  series  of  dependent  observations  is  called  time  series 
analysis.  Box-Jenkins  techniques  are  concerned  with  the  building  of  models  for 
discrete  time  series.  In  particular,  time  series  are  often  best  represented  by 
nonstationary  models  in  which  trends  and  other  pseudo-systematic 

characteristics  which  can  change  with  time  are  treated  as  stochastic  rather  than 
deterministic  phenomena.  Furthermore,  economic  time  series  often  possess  marked 
seasonal  or  periodic  components  themselves,  capable  of  change  and  needing 
(possibly  nonstationary)  seasonal  models  for  their  description.  Much  of  the 
motivation  behind  the  Box-Jenkins  approach  can  be  seen  through  reference  to 
Wold’s  decomposition  theorem.  This  theorem  demonstrates  that  any  covariance 
stationary  process  can  be  decomposed  into  two  parts,  r)t  and  where  r) t is  a 

purely  deterministic  part  and  c3 t is  the  purely  stochastic  part.  A corollary  of 
Wold’s  theorem  is  that  ? t can  be  expressed  as  a one-sided  moving  average  process 
of  infinite  order 


£t  ” at  ^lat-l  v^2at-2  " ••• 
or 

00 

" t = ' l ~ °jat-j 
j=o 

where  {at}  is  a sequence  of  uncorrelated  random  variables  and 

E(at)  = 0 for  all  t 
E(at)2  = for  all  t 

E(a|-as)  = 0 if  t£s 
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The  Box-Jenkins  approach  seeks  to  induce  stationarity  into  the  stochastic 
process  {Yt}  by  designing  filters  such  that 

dz  d|  X 

Zt  = As  A ( ( Yt+u ) -l)/\ 

is  a covariance  stationary  process  where 

A = Yt-Yt-i 

As  = Yt-Yt.s 
X 

((Yt+u)  -l)/\  = the  Box-Cox  power  transformation* 

The  mechanics  of  fitting  such  a model  (i.e.,  obtaining  estimates  for  dj , d2  and 
X)  involve  an  analysis  of  the  behavior  of  the  sample  autocorrelation  function 
and  the  sample  partial  autocorrelation  function.  With  respect  to  Wold's 
decomposition  theorem,  the  Box-Jenkins  approach  is  based  on  two  assumptions. 

d2  di 

The  first  assumption  is  that  the  differencing  operators,  As  A , and  the  power 
transformation  force  r\  t to  zero.  The  second  assumption  is  that 

£t  = <l>lEt-l  + •••  -N>pet-p+at“01at-l  “ •••  ” Qqat-q 


or 


4>(B)et  = Q(B)at 

P 

where  ( B ) = l-f})|B  - ...  - 4>qB 

q 

9(B)  = 1-9 iB  - ...  - 9 qB 
B = the  backshift  operator 

Both  assumptions  permit  the  infinite  number  of  parameters  in  the  original  model 
to  be  reduced  to  p+q+1  parameters,  p autoregressive  parameters,  q moving  average 
parameters,  and  the  variance  of  at,  aa.  The  previous  statement  shows  that  the 
types  of  models  produced  by  Box-Jenkins  techniques  have  a desirable  property: 
they  are  parsimonious.  Box-Jenkins  models  are  parsimonious  because  they  employ 
the  smallest  possible  number  of  parameters  for  adequate  representation. 
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APPENDIX  C:  THE  EVALUATION  OF  MULTI-MODEL  SYSTEMS1 


In  the  previous  sections  of  the  report,  the  focus  has  been  on  describing 
evaluation  considerations  as  they  apply  to  a single,  self-contained  model. 
However,  due  to  limitations  on  time  and  funds,  many  problems  encountered  by  FEMA 
can  only  be  studied  in  an  expedient  manner  by  combining  existing  models  into  a 
multi-model  system  in  which  the  separate  models  receive  data  from  and  transmit 
data  to  one  another.  For  such  systems,  the  analytic  aspects  of  evaluation, 
sensitivity  analysis,  data  audits,  and  so  on,  are  confounded  and  more  complex 
than  for  a single  model.  The  situation  is  further  complicated  as  the  individual 
models  that  are  combined  usually  have  evaluative  and  validation  problems  of 
their  own,  and  the  applicability  of  the  individual  model  is  stretched  to  meet 
the  requirements  of  the  multi-model  area.  In  such  situations,  validation 
testing  of  the  total  multi-model  system  is  not  possible;  such  testing  is  usually 
ignored.  The  model  analysts  must,  however,  provide  some  evidence  that  the 
multi-model  system  is  credible. 


The  main  criterion  of  credibility  is  the  notion  of  consistency.  In  any 
multi-model  study,  there  is  a requirement  to  establish  a set  of  initial 
conditions  — baseline  assumptions  — about  which  the  analysis  takes  place. 

This  requirement  exists  as  the  problem  setting  must  be  delineated;  the  models 
cannot  be  operated  without  fixing  the  many  input  parameters  and  initial  variable 
settings  that  define  the  problem  area.  A complete  statement  of  the  baseline 
assumptions  is  essential  in  a multi-model  analysis.  It  is  termed  the  baseline 
scenario.  Such  a statement  represents  the  major  control  mechanism  by  which  the 
models  and  analysts  are  coordinated  and  consistency  maintained  throughout  the 
analysis.  That  is,  a proper  baseline  scenario  ensures  that  the  models  (and 
analysts)  are  addressing  the  same  problem,  are  using  the  same  assumptions,  and 
are  starting  with  the  same  initial  data. 

A scenario,  in  general,  represents  a description  of  the  problem  area  being 
analyzed;  it  is  based  on  the  model  builders’  and  users’  perception  of  the 
problem  and  the  requirements  of  the  models.  Further,  a scenario  describes  the 
factual  and/or  postulated  settings  of  the  situation,  including,  in  model-based 
terms,  descriptions  of  the  future  in  which  the  analysis  takes  place.  Any 
model-based  analysis,  especially  a multi-model  analysis,  depends  upon  the 
construction,  definition,  interpretation  and  acceptance  of  the  baseline  and 
other  scenarios  by  which  the  multi-model  system  is  coordinated  and  comparative 
analyses  takes  place. 

Although  a scenario  represents  a statement  about  the  future,  it  cannot  be 
interpreted  as  a prediction,  but  only  as  a plausible  future  whose  implications 
are  to  be  analyzed  by  the  modeling  exercise.  Scenario-based  analyses  are 
conducted  by  varying  assumptions  and/or  data  to  determine  how  sensitive  the 
results  are  to  uncertain  estimates  of  the  future,  for  example,  measuring  the 
impact  of  uncertain  estimates  of  inflation  on  GNP.  Such  variations  usually  are 
conducted  about  a reference  or  baseline  scenario.  The  baseline  scenario  may  be 
considered  to  be  a description  of  the  status  quo  and  its  most  plausible 
projection  into  the  future.  The  baseline  scenario  thus  provides  the  reference 

"1  Portions  of  this  appendix  are  excerpted  from:  S.  I.  Gass,  and  S.  Parikh, 

"Credible  Baseline  Analysis  for  Multi-Model  Public  Policy  Studies,”  in  Ene rgy 
Models  and  Studies  B.  Lev  (ed.),  Nor th-Hol land , 1983. 
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conditions  that  are  input  to  the  multi-model  exercise.  It  enables  the  analysts 
to  produce  a scenario  impact  analysis  that  is  the  result  of  the  analysts 
translating  the  baseline  assumptions  into  corresponding  model  inputs,  the 
running  of  the  models,  and  output  analysis,  including  judgmental  adjustments  of 
model  results.  Taken  together,  the  baseline  scenario  and  the  resulting  impact 
analysis  form  the  reference  baseline  analysis. 

Some  uncertainty  usually  surrounds  many  of  the  assumptions  and  data 
specifications  of  a baseline  scenario.  There  is  a need  to  measure  the  impact  of 
this  uncertainty  by  performing  standard  sensitivity  studies  on  key  assumptions 
and/or  data  of  the  baseline  scenario.  This  can  be  an  expensive  and 
time-consuming  activity  and,  thus,  it  is  often  limited  to  just  a few  alterations 
of  the  baseline  scenario.  The  results  of  analyzing  these  alternate  baseline 
scenarios  represent,  in  a sense,  the  range  of  possible  outcomes  that  can  occur 
for  the  assumed  baseline  future.  For  example,  high  and  low  GNP  estimates  may  be 
run  and  compared  to  the  baseline  (average)  GNP,  while  keeping  all  other  input 
elements  constant.  In  many  studies,  the  above  type  of  analysis  is  the  extent  to 
which  a multi-model  system  is  employed. 


By  varying  the  assumptions  and  conditions  of  the  baseli 
the  views  of  the  plausible  future,  policy  impacts  can  be  ana 
of  possible  future  outcomes  and  compared  with  the  reference 
Then,  depending  on  their  assessment  of  the  most  likely  cours 
analysts  and  study  sponsors  can  account  for  the  uncertainty 
results.  The  impacts  will  often  change  with  the  scenario  as 
consistent  and  controlled  means  of  comparing  policy  implicat 
futures  is  provided  by  a systematic  use  of  the  baseline  and 
scenarios.  A comparative  analysis,  based  on  the  results  of 
plausible  scenarios,  puts  the  decision  maker  in  a better  pos 
which  policies  to  implement. 
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In  the  above  type  of  study,  in  which  a baseline  scenario  is  varied  to 
reflect  uncertainty,  the  set  of  choices  (policy  options)  available  to  the 
multi-model  system's  decision  process  is  not  changed.  Results  change  only 
because  a different  choice  is  made  under  the  alternate  baseline  scenarios. 
Another  and  probably  more  important  form  of  analysis  occurs  when  policy  options 
are  proposed  that  modify  the  baseline  policy  specifications.  The  analysis  then 
must  be  made  using  a set  of  choices  that  are  different  from  the  one  available  in 
the  baseline  analysis.  Alternate  policy  scenarios  are  then  used  to  produce 
policy  analyses  that  can  be  compared  to  the  baseline  analysis  to  measure  the 
impact  of  allowing  new  policy  options.  For  example,  a FEMA  baseline  scenario 
may  stipulate  that  all  steel  production  is  to  be  given  to  the  defense  tier,  with 
the  level  of  steel  production  being  an  uncertain  parameter  about  which 
sensitivity  studies  are  made.  But  an  alternate  policy  scenario  may  call  for 


138 


only  50%  of  the  steel  output  going  to  the  defense  tier.  Another  example  could 
have  the  baseline  scenario  requiring  mobilization  to  be  paid  for  by  taxation, 
while  an  alternate  policy  scenario  states  that  mobilization  would  be  financed  by 
deficit  spending.  The  alternate  policy  scenarios  are  subject  to  the  same 
uncera tinties  as  the  baseline  scenario.  To  conduct  a more  complete  policy 
study,  the  alternate  policy  should  be  varied  and  processed  in  the  same 
manner  as  the  alternate  baseline  scenarios.  However,  as  the  running  of  say, 
high,  average,  and  low  GNP  variations  for  each  scenario  is  costly  and  time 
consuming,  a study  is  usually  limited  to  just  the  processing  and  comparing  of 
the  average  baseline  and  policy  scenarios.  We  note  that  any  differences  in  the 
two  analyses  are  due  to  the  model’s  decision  logic  that  must  select  from  the 
different  policy  options  allowed  by  the  scenarios. 

Credibility  of  a multi-model  system  can  be  further  enhanced  if  the  reasons 
why  the  particular  set  of  models  was  selected  are  discussed.  For  example,  a 
model  that  has  been  used  in  the  past  with  favorable  results  and  does  not  need 
to  be  changed  contributes  positively  to  the  notion  of  credibility;  while  a newly 
built  model  or  one  that  must  undergo  extensive  reprogramming  will  not  inspire 
much  credibility.  No  matter  what  the  reasons  are,  they  need  to  be  made 
explicit.  Further  credibility  enhancing  elements  include  the  active  involvement 
of  the  user  in  specifying  the  scenarios  and  interpreting  the  results,  and  the 
use  of  experts  in  selecting  the  models  and  in  reviewing  the  results. 
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APPENDIX  D:  THE  POTENTIAL  ROLE  OF  DGEM  IN  AN  INTEGRATED  SYSTEM 

FOR  MILITARY  M06ILIZATI0N1 

There  are  several  agencies  within  the  Federal  government  responsible  for 
estimating  resource  requirements  in  the  event  of  a military  emergency.  Should 
an  actual  military  emergency  occur,  these  same  agencies  would  also  be 
responsible  for  allocating  available  resources  among  competing  ends.  The  most 
obvious  is  the  Department  of  Defense  (DoD)  which  would  be  responsible  for 
estimating  the  structure  of  the  military  forces  to  fulfill  the  requirements  of 
winning  or  preventing  a war.  This  would  include  estimates  of  dollar  outlays  to 
fulfill  the  force  structure.  The  Department  of  Commerce  (DoC)  has  the 
responsibility  to  evaluate  the  industrial  base  of  the  country  and  estimate  the 
output  of  goods  and  services  necessary  to  fulfill  not  only  the  requirements  of 
the  military,  but  also  the  requirements  of  the  civilian  economy.  This  task  also 
includes  estimates  of  production  required  for  industrial  investment  in 
structures  and  equipment,  net  exports,  state  and  local  government  expenditures, 
Federal  government  non-defense  expenditures  and  personal  consumption.  In  the 
production  of  output  for  investment  and  trade,  it  is  very  likely  that  the 
required  output  will  compete  with  production  for  national  defense.  Closely  tied 
to  the  Commerce  Department  estimates  of  output  by  industry  are  estimates  of  the 
labor  force  (employment  and  occupations).  This  task  is  performed  by  the 
Department  of  Labor  (DoL). 

Within  the  triangle  of  Federal  agencies  of  DoD,  DoC,  and  DoL  is  what  might 
be  considered  the  core  necessary  to  evaluate  the  industrial  production  and 
employment  effects  of  a military  emergency.  Within  each  of  these  agencies  is 
the  ability  to  estimate  total  production  and  .employment  needs  for  an  emergency. 
However,  the  responsibility  and  the  corresponding  capabilities  within  each 
separate  agency  to  analyze  its  resources  argues  for  cooperation  in  mobilization 
analysis.  As  a result,  these  three  agencies  signed  a Memorandum  of 
Understanding  in  1983  for  cooperation  in  mobilization  planning. 

In  addition  to  the  resources  monitored  by  DoD,  DoC,  and  DoL,  there  are 
other  Federal  agencies  responsible  for  monitoring  and  allocating  resources  in 
the  event  of  a crisis.  These  include:  (1)  energy  requirements  monitored  by  the 

Department  of  Energy  (DoE);  (2)  strategic  and  critical  materials  monitored  by 
the  Department  of  the  Interior  (Dol);  (3)  transportation  requirements  monitored 
by  the  Department  of  Transportation  (DoT);  and  (4)  food  and  agricultural 
requirements  monitored  by  the  Department  of  Agriculture  (DoA). 

This  appendix  outlines  how  these  executive  agencies  can  work  together,  and 
have  worked  in  the  past,  to  estimate  the  resources  necessary  to  meet  the 
requirements  of  a military  emergency. 

Central  to  the  discussion  in  this  appendix  is  the  role  of  macroeconomic 
models  in  interagency  mobilization  analyses.  Special  emphasis  is  placed  on  how 
DGEM  can  complement  proprietary  models  in  such  an  analytical  framework. 

-*-This  appendix  was  prepared  by  David  K.  Henry  and  Albert  Walderhaug  of  the 
Office  of  Business  Analysis,  Office  of  the  Under  Secretary  for  Economic  Affairs, 
U.  S.  Department  of  Commerce.  Several  contributions  by  Robert  E.  Chapman, 

Center  for  Computing  and  Applied  Mathematics,  National  Institute  of  Standards 
and  Technology  are  also  included. 
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This  appendix  does  not  attempt  to  concord  all  of  the  technical  parameters 
of  each  agency's  models  and  data.  To  concord  each  of  these  major  models  is  a 
considerable  task  and  would,  and  probably  should  as  in  past  efforts,  be 
accomplished  while  performing  an  interagency  mobilization  analysis.  At  that 
time,  details  such  as  the  industry  sectoring  classification,  constant  dollar 
series,  industry  to  commodity  conversion,  price  valuation,  and  national 
accounting  conventions  should  be  accommodated  and  made  consistent  among  the 
models  and  supporting  data  used  by  the  Federal  agencies  participating  in  the 
analyses.  The  constant  updating  and  change  in  the  technical  parameters  of  the 
models  that  can  be  used  in  interagency  analysis  is  another  reason  for  not 
attempting  to  concord  the  technical  parameters  in  this  appendix. 

The  most  recent  experience  of  coordinating  a multi-Federal  agency  effort 
analyzing  resources  to  meet  the  requirements  of  a military  emergency  was  an 
exercise  in  1984-85  that  was  coordinated  by  the  Federal  Emergency  Management 
Agency  (FEMA)  and  was  called  the  Federal  Resource  Assessment  System  (FRAS).  FRAS 
was  set  up  under  the  Military  Mobilization  Working  Group  of  the  Emergency 
Mobilization  Preparedness  Board  (EMPB).  The  Department  of  Commerce,  Office  of 
Business  Analysis  (OBA)  was  responsible  for  the  technical  coordination  of  all 
the  participating  Agencies.  The  effort  resulted  in  a report  to  EMPB  in  1985 
which  described  the  interagency  effort  and  provided  preliminary  estimates  of  the 
resources.  The  EMPB  dissolved  shortly  thereafter.  Since  then,  a Senior 
Interagency  Group  (SIG)  for  National  Security  Emergency  Planning  has  been 
established  under  the  National  Security  Council.  The  Mobilization  Planning 
Group  under  the  SIG  has  been  an  attempt  to  continue  an  interagency  governmental 
responsibility  in  mobilization  planning. 

Also  in  1984-85,  the  National  Security  Council  led  an  effort  to  evaluate 
the  quantities  of  strategic  and  critical  materials  that  should  be  maintained  in 
the  U.  S.  government's  stockpiles  in  the  event  of  a military  emergency.  The 
stockpile  study  used  a similar  approach  to  the  FRAS  analysis,  since  industrial 
output  requirements  during  a military  emergency  needed  to  be  estimated  before 
the  estimate  of  critical  materials  could  be  made.  The  stockpile  review  was  also 
an  interagency  effort  and  a good  portion  of  its  military  emergency  assumptions 
were  incorporated  into  the  FRAS  analysis. 

In  addition  to  the  FRAS  and  the  Stockpile  interagency  efforts  to  analyze 
the  requirements  of  a military  emergency,  there  was  an  interagency  effort  to 
evaluate  the  effects  of  imports  on  the  machine  tool  industry  for  a Section  232 
investigation  under  the  1962  Trade  Expansion  Act  for  Import  Vulnerability.  The 
results  of  the  study  led  to  the  decision  to  apply  Voluntary  Restraint  Agreements 
on  the  imports  of  machine  tools  into  the  United  States. 

Each  of  these  studies  required  an  evaluation  of  the  resources  of  the  entire 
economy.  Military  and  civilian  resource  requirements  estimates  were  made  for 
products  common  within  each  industry  and  decisions  were  made  about  the 
industry's  ability  to  meet  these  requirements.  In  each  case,  industrial  output 
was  estimated  to  fulfill  the  production  of  goods  and  services  that  were 
generated  in  an  economy  described  by  a level  and  composition  of  Gross  National 
Product  (GNP).  The  economy  as  described  by  its  GNP  was  used  as  the  first  step 
in  the  analysis. 
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Figure  D.l  Scenario  Development  and  Analysis 
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The  Economy  Under  A Mobilization 


Implicit  in  any  military  mobilization  preparedness  study  is  a well-defined 
set  of  models,  databases,  and  analysis  strategy.  Figure  D.l  illustrates  the 
process  used  to  describe  the  domestic  economy  during  an  emergency. 


The  first  stage  of  any  such  study  is  the  specification  of  a scenario.  A 
scenario  is  defined  as  an  account  or  a synopsis  of  a projected  course  of  actions 
or  events.  The  military  scenario  describes  a course  of  actions  or  events  which 
includes  an  enemy  threat,  a basic  strategy  to  meet  the  threat,  the  size  and 
deployment  of  U.S.  forces  that  fulfills  the  military  strategy  and  the  form  and 
intensity  of  the  combat  expected.  In  turn,  the  size  and  deployment  of  U.S. 
forces  requires  a defense  budget  that  accommodates  or  allows  their  support. 


In  addition  to,  or  as  a result  of,  the  military  scenario,  there  is  an 
economic  scenario  which  describes  how  the  domestic  economy  reponds  to  the 
military  emergency.  In  developing  the  economic  scenario,  policy  decisions 
affect  the  way  the  economy  reacts  to  meet  the  threat.  Economic  assumptions 
reflect,  for  example,  the  monetary  and  fiscal  policies  chosen  to  finance  the 
war,  the  world  energy  supply  and  corresponding  prices,  and  projections  for  the 
expected  rate  of  growth  of  the  economy  and  the  Federal  budget.  Once  the  economy 
has  been  described,  in  quantitative  terms,  it  is  possible  to  estimate  the 
production  of  goods  and  services  to  meet  the  requirements  of  the  economy. 


The  need  for  macroeconomic  (economy  wide)  analysis  requires  that  models  of 
the  economy  be  used  to  determine  the  economy's  size  and  composition.  There  are 
a number  of -proprietary  models  such  as  those  marketed  by  Wharton,  Chase  and  DRI 
available  to  be  used  by  government  agencies.  In  addition,  there  are  some  models 
developed  within  the  government  for  special  purpose  analyses,  which  incorporate 
economy  wide  analysis  as  a front-end  to  their  special  analyses:  the  Defense 

Economic  Impact  Modeling  System  (DEIMS),  used  by  DoD  to  assess  the  market  share 
of  output  for  defense;  the  Economic  Growth  System  (EGS)  of  the  Bureau  of  Labor 
Statistics  (BLS),  used  to  analyze  future  needs  for  labor  and  skills  as  the 
economy  grows  and  changes;  as  well  as  DGEM . 

The  proprietary  models  are,  in  general,  peacetime  models  based  on  peacetime 
trends  and  relationships.  Their  suitability  for  mobilization  analysis  depends 
on  the  extent  to  which  the  structural  relationships  in  the  model  can  accommodate 
policy  relevant  variables  dictated  by  the  mobilization  scenario  and  the  economic 
objectives  which  are  subject  to  the  set  of  policy  options.  The  options  relate 
to  how  the  economy  will  be  managed  to  achieve  the  mobilization  objectives  and 
will  involve:  (1)  tax  policy  and  how  it  relates  to  persons  and  business;  (2) 

fiscal  policy  as  it  determines  how  the  emergency  is  financed;  (3)  monetary 
policy  affecting  the  growth  of  money  supply  and  its  impact  on  interest  rates; 
and  (4)  government  actions  affecting  labor  force  participation  rates  and 
producitivity  improvement  incentives. 


Since  econometric  models  are  based  on  historical  relationships  and  trends, 
it  is  important  that  they  incorporate  and  reflect  up-to-date  information  to  the 
maximum  extent  possible.  The  suitability  of  models  from  this  aspect  needs  to  be 
assessed.  Furthermore,  the  release  by  the  Bureau  of  Economic  Analysis  of  the 
Department  of  Commerce  of  the  1977  input-output  tables  and  rebenchmarking  of  the 
National  Accounts  to  1982  prices,  which  form  the  basis  of  the  macroeconomic 
models  and  derivative  models,  may  cause  some  problems  of  price  compatibility  and 
industry  classification  and  structural  relationships. 
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Recently,  several  Federal  agencies  have  expressed  interest  in  using  DGEM  to 
complement  their  modeling  efforts.  For  example,  DGEM  may  be  used  to  forecast 
the  values  of  key  macroeconomic  variables  (e.g.,  GNP , Personal  Consumption 
Expenditures,  Investment,  as  well  as  other  components  of  the  National  Income  and 
Product  Accounts  (NIPA))  which  may  be  needed  as  inputs  for  a particular  model. 
Clearly,  experiences  gained  by  these  Federal  agencies  would  be  beneficial  in 
performing  similar  analyses  with  DGEM  for  a war-time  economy. 1 

DGEM  may  be  useful  in  this  role  for  four  other  reasons.  First  and 
foremost,  DGEM  was  designed  to  deal  with  the  demand  surges  and  resource 
constraints  which  are  expected  during  the  transition  from  a peace-time  to  a 
war-time  economy.  Specifically,  DGEM  explicitly  addresses  factors  which  affect 
both  demand  and  supply  on  an  annual  basis  up  through  the  year  2000.  Second,  the 
TUTOR  provides  the  user  with  a set  of  step-by-step  instructions  for 
incorporating  information  on  Defense  and  other  government  expenditures  as  well 
as  economic  and  technical  considerations  which  specify  how  the  economy 
mobilizes.  Third,  from  a review  of  the  documentation,  it  became  clear  that  much 
care  went  into  the  design,  development  and  testing  of  the  model.  Furthermore, 
the  model's  documentation  is  quite  complete  and  should  be  sufficient  to:  (1) 

set  it  up  on  the  host  system;  (2)  execute  a "base-case"  simulation;  (3) 
interpret  the  results  of  the  "base-case’  simulation;  and  (4)  create,  run,  and 
interpret  user-specified  simulations.  Finally,  the  DGEM  source  code  has  been 
made  available  to  a number  of  users,  enabling  them  to  explore  the  major  linkages 
within  the  model  as  well  as  trace  the  flow  of  calculations  for  particular  (e.g., 
mobilization  oriented)  variables  of  interest. 

These  "strengths"  must  be  weighed  against  the  lack  of  sector-specific 
detail  in  DGEM.  At  the  present  time,  DGEM  consists  of  36  producing  industries. 
More  precisely,  the  DGEM  analysis  framework  includes:  (1)  inputs  to  each 

industry  from  each  of  the  36  producing  industries  and  from  capital,  labor  and 
competitive  imports;  and  (2)  outputs  from  each  industry  to  each  of  the  36 
producing  industries  and  to  final  demand  — personal  consumption,  investment, 
government  purchases  (specifically,  Federal  defense,  Federal  nondefense,  and 
state  and  local  government),  and  exports.  Bridge  tables,  either  within  DGEM  or 
between  the  TUTOR  and  DGEM,  are  of  crucial  importance  for  most  military 
applications.  The  reasoning  behind  the  previous  statement  is  that  bottlenecks 
are  more  likely  to  occur  as  the  sectoral  level  of  detail  is  increased. 
Conversely,  the  higher  the  level  of  aggregation,  the  less  likely  is  the  analyst 
to  be  able  to  identify  a potential  bottleneck.  Based  on  the  previous 
discussion,  it  should  be  clear  that  bottlenecks  within  DGEM's  36  sector 
framework  can  be  identified,  analyzed  and  to  a certain  extent  alleviated.  The 
question  thus  becomes  one  of  sectoral  detail.  It  is  also  worth  noting  that  the 
sectoral  level  of  detail  of  DGEM  differs  significantly  from  DEIMS  (400  sectors) 
and  the  BEA  benchmark  input-output  table  (537  sectors).  This  lack  of  sectoral 
detail  is  probably  the  most  serious  deficiency  of  DGEM  with  regard  to  military 
mobilization  applications. 

Based  on  our  review  of  DGEM , both  the  model's  source  code  and  documentation 
and  the  TUTOR,  it  is  clear  that  the  addition  of  two  bridge  tables  would 
substantially  enhance  the  ability  of  DGEM  to  provide  a detailed  analytical 


1 For  a more  detailed  discussion  of  DGEM,  the  TUTOR,  the  DGEM  analysis  strategy, 
and  other  related  topics,  the  interested  reader  is  referred  to  Chapters  6 
through  9. 
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framework  for  a wide  variety  of  military  mobilization  applications.  The  first 
is  a "pre-processor”  which  incorporates  a bridge  into  the  TUTOR  between  the  55 
defense  procurement  categories  used  in  DEIMS  and  DGEM 1 s 36  producing  sectors. 
Alternatively,  the  "pre-processor"  bridge  table  could  be  between  DoC's  defense 
procurement  categories  list  and  DGEM ' s 36  producing  sectors.  In  the  text  which 
follows  some  of  the  pros  and  cons  of  both  DoD ' s and  DoC's  bridge  tables  between 
their  respective  defense  procurement  categories  and  the  economy's  producing 
sectors  are  discussed.  The  second  is  a "post-processor"  which  translates  the 
DGEM  output  into  a form  which  is  consistent  with  DoC's  4-digit  Standard 
Industrial  Classification  (SIC)  level.  If  this  were  done,  the  output  of  DGEM 
would  be  consistent  with  the  BEA  benchmark  input-output  table.  We  note  that  the 
"pre-processor"  bridge  table  discussed  earlier  may  be  incorporated  in  a 
relatively  straight-forward  manner.  This  is  because  much  of  the  logic  has 
already  been  incorporated  into  the  TUTOR.  Recall  that  government  purchases  are 
separated  into  three  types'  of  government  activity:  (1)  Federal  defense;  (2) 

Federal  nondefense;  and  (3)  state  and  local  government.  In  turn,  each  type  of 
government  activity  involves  purchases,  spread  over  a number  of  supplying 
industries.  For  Federal  defense  purchases,  a rudimentary  bridge  table,  which 
distributes  defense  purchases  from  the  55  DoD  procurement  categories  to  the  36 
producing  sectors,  is  included  in  the  TUTOR.  The  analyst  can  therefore  control 
defense  spending  in  a variety  of  ways  within  this  structure.  Overall  levels  of 
defense  spending  can  be  altered,  leaving  the  composition  unchanged,  or  the 
composition  and  level  can  be  changed.  Specific  defense  procurement  categories 
can  be  altered,  year  by  year.  Federal  nondefense  purchases  and  state  and  local 
government  purchases  are  spread  over  the  36  producing  sectors.  Federal 
nondefense  purchases  can  also  be  altered  in  total  or  industry  by  industry. 

State  and  local  purchases  can  be  altered  in  total,  although  the  composition 
remains  the  same.  The  development  of  a "post-processor"  bridge  table  is 
somewhat  more  complex,  but,  in  principle,  most  of  the  data  exists  to  go  from  the 
NIPA  components  to  the  4-digit  SIC  level  of  detail.  If  these  steps  were  taken, 
an  analysis  framework  similar  to  that  outlined  in  Figures  D.l  through  D.3  and 
described  in  the  text  which  follows  would  result. ^ 

To  summarize,  the  key  outputs  required  from  the  macroeconomic  model  are  the 
"snapshots"  of  the  U.S.  economy  for  the  years  in  which  the  economy  is  distorted 
by  a mobilization  or  a military  emergency.  The  description  of  the  economy 
needed  in  order  to  evaluate  industrial  requirements  and  supply  conditions  are 
the  levels  of  real  GNP  and  its  composition. 

Estimating  Industrial  Production  and  Supply 


Figures  D.2  and  D.3  illustrate  the  process  generally  followed  in  analyzing 
the  industrial  impacts  of  major  government  programs,  or  responses  to  emergency 
situations.  Within  the  parameters  of  GNP  and  its  components,  the  output  of 
goods  and  services  implied  by  the  scenario  are  estimated  as  well  as  the 
capability  of  the  existing  economy  to  fulfill  the  requirements. 


^It  is  important  to  point  out  tha 
on  DGEM  as  complementary  to  those 
It  is  neither  reasonable  nor  desi 
supplant  a loosely-coupled  collec 
but  rather  to  use  such  a model  to 
feedback  within  the  overall  analy 


t we  view  such  an  analysis  framework  based 
based  on  collections  of  models  and  database 
rable  to  have  a single  model,  such  as  DGEf , 
tion  of  models  from  several  Federal  agencies 
encourage  consistency  and  an  avenue  for 
tical  framework. 
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Figure  D.2  Analysis  of  Production  Requirements 
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Figure  D.3  Analysis  of  Capacity 
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OBA  has  developed  bridge  tables  to  be  used  in  translating  the  final  demand 
components  of  GNP  into  their  industrial  composition.  These  estimates  show  the 
direct  impact  of  the  final  demand  for  goods  and  services  on  the  producing 
industries  of  the  economy.  However,  before  the  bridge  tables  can  be  used,  a set 
of  "control"  totals  must  be  established  that  correspond  to  the  detail  of  the 
bridge  tables.  A macroeconometric  model  is  used  to  establish  the  levels  for 
most  of  the  totals  (composition  of  GNP).!  However,  as  appropriate,  in  all  of 
these  mobilization  exercises  the  level  and  composition  of  national  defense 
spending  deserves  special  attention. 


Military  Requirements 

One  of  the  most  sensitive  issues  to  be  addressed  by  mobilization  planners 
is  the  size  and  the  composition  of  the  military  budget  to  meet  the  threat. 
Although  the  total  national  defense  budget  is  a key  exogenous  variable  in  the 
macroeconomic  analysis  of  the  economy,  the  composition  of  national  defense  is 
necessary  to  allocate  industrial  final  demand  to  the  defense  expenditures.  For 
the  FRAS , Stockpile  Review  and  the  Section  232  Machine  Tool  studies,  the  detail 
of  the  military  budget  estimates  by  budget  category  have  been  supplied  by  the 
Defense  Department. 

An  alternative  way  to  translate  a military  scenario  into  defense  outlays  by 
budget  category  has  been  through  a model  developed  by  The  Analytical  Sciences 
Corporation  (TASC)  for  FBI  A called  TASCMAIN,  or  Technique  for  Assessing  the 
Capability  to  Mobilize  American  Industry.  Although  TASCMAIN  was  designed  to 
evaluate  industrial  supply  and  demand  during  a mobilization,  its  uniqueness  for 
an  interagency  exercise  is  its  ability  to  translate  a military  scenario  into  a 
force  structure  which  is,  in  turn,  translated  into  DoD  budget  accounts  in 
dollars.  These  budget  accounts  are  necessary  for  DoC  to  calculate  military 
demand  on  industry. 


Industrial  Final  Demands  for  Military  Requirements 

The  emphasis  of  the  work  conducted  in  OBA  to  translate  the  GNP  components 
of  final  demand  into  their  industrial  composition  has  been  on  Federal  government 
expenditures  for  national  defense.  OBA  has  developed  34  bridge  tables  which 
translate  DoD  budget  expenditures  into  their  industrial  composition.  These 
bridge  tables  are  keyed  to  defense  budget  categories:  military  personnel; 

operations  and  maintenance;  procurement;  research,  development,  testing  and 
evaluation;  military  construction;  family  housing  and  non-DoD  defense  programs. 

An  alternative  to  the  OBA  translation  of  the  defense  budget  to  industrial 
final  demands  is  by  the  use  of  the  defense  budget  translators  maintained  by  the 
Department  of  Defense’s  Program,  Analysis  and  Evaluation  Division.  The 
translators  are  a part  of  the  Defense  Economic  Impact  Modeling  System  (DEIMS) 
and  in  a great  many  respects  are  similar  to  the  translators  or  bridge  tables 
maintained  by  the  Department  of  Commerce. 

1-0 BA  is  currently  exploring  a number  of  ways  in  which  DGEM  may  be  used  to 
generate  the  control  totals  required  in  defense-related  studies. 
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The  Civilian  Economy 


Estimates  of  components  of  GNP  resulting  from  the  macroeconomic  analysis, 
in  addition  to  Federal  government  expenditures  for  national  defense,  include 
expenditures  for  personal  consumption,  gross  private  capital  formation,  Federal 
expenditures  for  non-defense,  state  and  local  government  expenditures,  exports 
and  imports.  During  peacetime,  the  percentage  of  GNP  accounted  for  by  national 
defense  has  ranged  from  5 to  6 percent  in  recent  years.  For  each  of  the 
civilian  components  of  GNP,  bridge  tables  were  developed  that  describe  their 
composition  of  industrial  final  demand.  The  expenditure-to-commodity  relation- 
ships developed  in  the  bridge  tables  reflect  historical  relationships. 

Therefore,  the  greater  the  GNP  expenditure  detail  available  from  the 
macroeconomic  model,  the  more  flexibility  there  can  be  in  using  bridge  tables  to 
specify  a "bill-of-goods"  to  describe  the  industrial  composition  of  non-defense 
GNP  or  final  demand. 


Sub-Tier  Production 

Final  demand  does  not  reflect  the  intermediate  goods  and  services  which  are 
consumed  in  making  end  products  that  are  purchased  for  national  defense  or  the 
rest  of  the  economy.  A further  step  is  necessary  to  estimate  the  output  of 
industries  that  is  required  to  supply  those  indirect  products,  as  well  as  the 
products  entering  final  demand  directly.  This  is  accomplished  using 
input-output  techniques  in  which  the  bills-of-goods  (aircraft,  missiles,  tanks, 
ships,  etc.)  are  applied  to  an  input-output  inverse  matrix  which  generates  the 
prime  contractor  and  sub-tier  contractor  requirements  for  output  by  industry  to 
support  the  specified  level  and  detail  of  GNP.  The  levels  of  total  output, 
direct  and  indirect,  by  industry  resulting  from  these  operations  form  the  basis 
for  subsequent  evaluations  of  the  adequacy  of  resources  to  meet  the 
requirements . 

The  input-output  models  of  the  U.S.  industrial  economy  available  for  use  in 
estimating  the  indirect  output  by  industry  to  fulfill  the  final  demand  for  goods 
and  services  required  by  the  emergency  are  derived  mainly  from  the  537-sector 
interindustry  tables  produced  by  DoC's  Bureau  of  Economic  Analysis  (BEA).  These 
derivative  tables  are  condensed  and  in  some  cases  updated  versions  of  the  BEA 
tables  with  industry  classifications  designed  for  specific  purposes.  The  latest 
version  of  the  BEA  input-output  tables  are  for  1977.  The  next  revision  of  the 
benchmark  table  will  be  for  1982,  although  revisions  for  1981  and  1982  of  the 
more  aggregated  80-industry  order  table  have  been  produced.  Other  interindustry 
input-output  tables,  derived  from  the  BEA  benchmark  table,  include:  the  DRI 

interindustry  model  (432  sectors),  the  BLS  Economic  Growth  System  (156  sectors), 
the  FEMA  model  called  the  Revised  Growth  for  Industrial  Potential,  REGRIP, 

(115  sectors),  and  DoE's  Energy  Disaggregated  Input-Output  Model  (130  sectors). 

Industrial  Capacity 

Estimates  of  industrial  capacity  by  which  to  assess  the  feasibility  of 
producing  the  output  as  required  during  the  emergency  has  and  continues  to  be 
the  weakest  link  in  industrial  analysis.  The  Bureau  of  the  Census  report 
entitled  Survey  of  Plant  Capacity  lists  capacity  utilization  rates  for  most 
manufacturing  industries  at  the  4-digit  Standard  Industrial  Classification  (SIC) 
level.  There  are  some  considerable  problems  in  utilizing  the  Census  data  for 
mobilization  planning.  The  utilization  rates  are  for  fourth  quarter  activity 
only  and  are  not  average  annual  activity.  If  there  was  only  one  shift  operating 
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during  the  fourth  quarter,  estimated  capacity  is  based  on  that  one  shift. 

During  a military  emergency,  double  and  triple  shifts  could  occur  and  the 
industry's  capacity  would  increase  accordingly.  This  increase  would  be  offset 
somewhat  by  the  faster  rate  of  capital  stock  depletion. 

FEMA  recognized  the  problems  in  using  industrial  capacity  data  for 
mobilization  planning.  Consequently,  they  contracted  ORI,  Inc.  to  provide 
estimates  of  emergency  capacity  for  manufacturing  industries.  The  estimates  for 
emergency  capacity  for  mining,  manufacturing,  and  utility  industries  include 
labor  and  capital  shift  factors  and  lead  times  for  "greenfield"  investments. 

The  emergency  capacity  data  developed  by  ORI,  Inc.  can  be  used  in  conjunction 
with  the  normal  sources  of  capacity  data  from  the  Census  and  the  Federal  Reserve 
Board. 


In  the  past,  DoC  conducted  Industry  Evaluation  Board  (IEB)  studies  to 
determine  the  capacities  of  defense-related  industries.  These  capacity  studies 
were  done  at  a very  high  level  of  detail.  The  IEB  capacity  studies  used 
considerable  investigative  resources  and  took  a long  time  to  complete.  Survey 
forms  and  mailing  lists  were  prepared  and  surveys  mailed.  Follow-ups  were  made 
to  non-responders  and  the  responses  were  reviewed  and  analyzed. 

More  recently,  OBA,  at  the  request  of  the  National  Security  Council  and  the 
Office  of  Management  and  Budget,  developed  a methodology  to  assess  the  invest- 
ment requirements  of  certain  manufacturing  industries  to  meet  the  industrial 
outputs  projected  for  military  mobilization. 

The  methodology  to  estimate  new  and  replacement  investment  of  the 
industries  evaluated  depend  on  OBA's  data  base  on  capital  stocks  of 
manufacturing  industries.  The  OBA  capital  stock  data  base  consists  of  measures 
of  investment,  capital  stocks  and  related  statistics  for  manufacturing 
industries.  Twenty-one  measures  of  capital  stocks  for  structures  and  equipment 
are  available. 

In  OBA's  method  for  estimating  new  and  replacement  investment,  ratios  of 
capital  stocks  per  unit  of  output  are  developed  for  each  of  the  industries. 

Based  on  the  requirements  estimates,  a determination  can  be  made  from  these 
ratios  as  to  the  level  of  capital  stocks  necessary  to  support  the  industrial 
output  requirements  during  the  emergency.  Comparing  existing  capital  stocks  to 
the  emergency  capital  stocks  requirements,  and  factoring  the  physical 
depreciation  (discards  and  wear  and  tear)  of  the  existing  capital  stocks  into 
the  emergency  period,  the  amount  of  new  investment  can  be  calculated. 

If  the  industrial  output  projection  for  defense  related  industry  exceeds 
existing  capacity,  a decision  must  be  made  to  provide  or  not  provide  for  the  new 
investment.  If  not  provided  for,  the  industry  that  needed  the  investment  would 
have  to  be  identified  as  a bottleneck  industry  — an  industry  unable  to  supply 
the  estimated  military  and  civilian  requirements  and  an  alternative  policy 
decision  on  allocation  of  available  supply  would  have  to  be  made. 

The  investment  in  the  industries  needed  to  meet  the  emergency  production 
levels  is  then  distributed  to  the  types  of  equipment  and  structures  that  are 
necessary  to  provide  the  facilities.  For  example,  machine  tools  and  testing  and 
measuring  equipment  are  important  in  the  production  of  weapon  systems.  OBA 
maintains  a capital  flow  matrix  that  identifies  the  types  of  equipment  and 
structures  in  an  industry's  investment  pattern. 
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These  calculations  of  investment  requirements  really  redefine  the  previous 
estimates  of  Gross  Private  Fixed  Investment  which  have  been  estimated  in  a 
less  refined  fashion.  In  other  words,  the  GNP  is  redefined  and  industrial 
requirements  of  all  industries  are  restructured  and  additional  output  will  be 
required  from  an  undefined  set  of  industries  to  meet  this  investment.  Thus,  a 
second  or  even  third  review  of  industrial  capacity  may  be  necessary. 

Recently,  FEMA  developed  a model  to  identify  capacity  shortfalls  for 
industrial  mobilization  and  the  options  to  resolve  them.  The  model  will  be  used 
by  the  Army  Material  Command  to  analyze  industrial  preparedness.  The  model  was 
named  ROCS  (Resolution  of  Capacity  Shortfalls).  The  primary  advantages  of  ROCS 
are:  (1)  that  it  can  be  implemented  on  a Personal  Computer  (PC);  (2)  that  it 

calculates  the  cutbacks  in  civilian  consumption  necessary  to  meet  military 
requirements;  and  (3)  offers  alternatives  to  fulfill  supply  requirements. 

The  model  utilizes  most  of  the  data  used  for  the  REGRIP  model  except  that, 
while  REGRIP  differentiates  only  between  civilian  consumption  and  military 
requirements,  ROCS  deals  with  more  detailed  categories  of  civilian  consumption: 
(1)  personal  consumption  expenditures;  (2)  investment;  (3)  exports;  (4) 
non-military  Federal  government  purchases;  and  (5)  state  and  local  government 
purchases.  ROCS  also  differs  from  REGRIP  in  that  an  involved  linear  programming 
solution  is  not  required.  Another  feature  of  ROCS  is  the  inclusion  of  data  on 
the  sources  of  imports  from  18  regions  of  the  world;  the  user  may  supply  their 
own  estimates  of  import  percentages  and  reliability  for  each  region. 


Estimating  Employment  Requirements  and  Availability 

DoL  is  responsible  for  estimating  and  evaluating  all  aspects  of  the  U.S. 
labor  force  for  a military  emergency.  This  includes  estimating  both  defense  and 
civilian  labor  requirements  by  specific  industry  and  occupation,  projecting  the 
labor  force  available,  and  determining  the  feasibility  of  mobilization  scenarios 
in  terms  of  labor  resources. 

The  Economic  Growth  System  (EGS)  of  the  Bureau  of  Labor  Statistics  (BLS)  is 
designed  to  project  the  military  and  civilian  labor  force  by  age  and  sex,  labor 
productivity  and  labor  requirements  for  378  industries  and  550  occupations. 

This  system  is  used  to  estimate  labor  requirements  that  are  consistent  with 
macroeconomic  assumptions  and  conditions  and  industry  production  levels 
estimated  by  DoC.  The  EGS  model  overlaps  the  macroeconomic  models  used  to 
predict  the  economy  and  production  estimates  made  by  DoC.  Its  uniqueness  in  the 
interagency  framework  is  the  employment  and  occupations  matrices  incorporated 
into  the  EGS  model. 

The  EGS  contains  the  Labor  Force  Model  which  projects  the  labor  force  based 
on  projections  made  by  the  Bureau  of  the  Census  of  "total  population  including 
Armed  Forces  overseas."  The  population  estimates  are  grouped  by  age,  sex,  and 
race.  Labor  force  participation  rates  are  projected  by  BLS  for  each  of  54 
demographic  groups  and  applied  to  the  population  data  to  estimate  the  aggregate 
labor.  This  technique  provides,  in  effect,  an  estimate  of  capacity  in  terms  of 
labor  resources  against  which  to  evaluate  the  demand  for  labor  as  determined 
from  estimates  of  output  by  industry  that  will  be  required  in  the  mobilization 
effort.  Undoubtedly,  in  developing  the  scenario  for  the  emergency,  personnel 
requirements  for  the  armed  forces  would  have  an  impact  on  projection  of  the 
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civilian  labor  forces  and  on  projections  of  potential  GNP  when  formulating  the 
macroeconomic  model.  Similarly,  the  demographic  detail  of  the  population  and 
labor  force  participation  rates  will  give  guidance  in  policy  decisions  dealing 
with  the  perceived  labor  shortages  in  the  resource  evaluation  procedure. 

The  BLS  Economic  Growth  System  contains  an  input-output  model  that  is  used 
to  convert  estimates  of  the  level  of  GNP  and  its  components  into  estimates  of 
gross  output  for  156  industries.  The  estimated  industry  outputs  are  used 
to  project  employment  by  industry  using  a Labor  Demand  Model.  However,  for 
consistency  in  the  interagency  program,  the  EGS  can  accept  estimates  of  gross 
output  by  industry  for  the  overall  resource  assessment  system  which  would  be  in 
greater  industry  detail.  When  aggregated  to  the  classification  system 
consistent  with  the  Labor  Demand  Model,  gross  output  by  industry  can  be 
converted  into  related  employment  by  industry  and  then  aggregated  to  assess 
total  requirements  for  civilian  employment. 

The  Economic  Growth  System  also  contains  an  Occupational  Demand  Model 
which  uses  industry  occupational  staffing  patterns  to  estimate  the  skill 
requirements  of  industries  for  550  occupations  which,  when  aggregated,  will 
yield  estimates  of  total  employment  by  occupation. 

Estimating  Infrastructure  Requirements 

DoE  is  the  resource  agency  charged  with  energy  resource  management 
functions.  In  providing  analytical  capability  in  support  of  these  functions, 
the  Energy  Information  Administration  maintains  a complete  hierarchy  of  models 
and  modeling  systems  to  evaluate  alternative  aspects  of  energy  supply  and  demand 
to  provide  the  basis  for  policy  decisions  within  the  DoE.  This  analytical 
capability  can  be  called  upon  in  support  of  an  interagency  resource  assessment 
plan  to  examine  the  energy-related  aspects  of  a military  mobilization. 

The  Energy  Disaggregated  Input-Output  (EDIO)  model  is  an  energy  oriented 
input-output  system  designed  to  expand  the  Department’s  capability  to  analyze 
the  potential  industrial  economic  impacts  of  energy  policies.  In  presenting  a 
more  systematic  representation  of  the  basic  energy  interrelationships  in  the 
economy,  the  EDIO  model  permits  a much  more  detailed  analysis  of  the  energy 
requirements  within  an  1-0  framework. 

The  sectoring  plan  in  EDIO  is  important  since  it  defines  the  industry  and 
commodity  detail.  While  the  industry  classification  used  in  EDIO  can  be  related 
to  the  other  1-0  systems  available  for  mobilization  analysis,  the  basic  focus  of 
the  EDIO  table  is  energy,  and  therefore,  the  EDIO  industry  classification 
emphasizes  energy  products.  The  model  disaggregates  energy  producing 
industries,  industries  consuming  large  amounts  of  energy  per  unit  of  output  from 
other  sectors  of  the  economy,  thus  enabling  the  analyst  to  identify  explicitly 
those  industries  most  likely  to  be  affected  by  energy  policy.  These  features 
provide  a model  capable  of  identifying  the  sectoral  economic  impacts  for 
different  energy  and  macroeconomic  scenarios.  On  the  other  hand,  in  other  areas 
of  the  economy  where  energy  is  not  particularly  important,  the  industries  have 
been  combined  to  yield  a smaller,  more  manageable  system  but  one  with 
considerably  more  energy  detail. 

There  are  two  approaches  that  can  be  taken  in  utilizing  the  EDIO  model  in 
an  interagency  resource  assessment  system.  The  model  itself  is  essentially  self 
contained  requiring  as  inputs  components  of  gross  national  product  from  a 
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macroeconomic  model.  This  requirement  would  be  satisfied  in  the  general  plan  of 
the  overall  assessment  system  which  employs  such  a model  in  developing  the 
mobilization  scenario.  The  EDIO  system  has  a submodel  which  disaggregates  the 
GNP  components  into  their  industrial  composition  in  order  to  compile  a vector  of 
final  demand  in  the  industrial  detail  compatible  with  the  EDIO  models  i.e.,  a 
bill-of-goods  representing  sales  to  users  by  each  of  the  117  sectors  of  the 
model.  Total  gross  output  by  industry  to  satisfy  a given  bill-of-goods  is 
derived  through  multiplication  of  the  EDIO  inverse  matrix  and  the  bill-of-goods. 

Another  approach  to  the  assessment  of  energy  requirements  during  the 
mobilization  recognizes  that  a central  and  coordinating  element  to  the  resource 
assessment  system  is  the  step  which  estimates  the  gross  output  by  industry 
required  to  support  the  mobilization  effort.  This  calculation  is  carried  out 
using  an  1-0  model  with  much  greater  industrial  detail  (i.e.,  the  500-order 
indus try/commodity  tables  produced  at  DoC  by  BEA) . The  industry  detail  of  gross 
output  from  these  estimates  can  be  aggregated  to  the  level  of  industrial  detail 
consistent  with  that  maintained  at  DoE.  Energy  output  coefficients  from  the 
EDIO  model  or  from  the  National  Energy  Accounts  (NEA)  system  can  then  be  applied 
to  determine  energy  requirements. 

In  the  end,  this  may  be  a more  flexible  approach  since  the  energy  inputs 
are  perhaps  some  of  the  more  variable  inputs  to  industry  and,  within  limits, 
alternative  configurations  of  energy  sources  and  allocations  can  be  evaluated 
without  significant  feedback  effects  that  would  alter  the  output  required  of  the 
rest  of  the  producing  industries. 


TRANSPORTATION 

DoT  has  developed  and  utilizes  a demand/capacity  model  which  translates  a 
given  level  of  the  U.S.  economy  into  an  estimated  demand  on  the  domestic 
transportation  sector. 1 This  model  has  the  capability  to  address  both  the 
overall  mobilization-related  transportation  demand/capacity  requirements  and 
also  the  potential  impact  on  the  transportation  infrastructure  caused  by 
physical  denial  and/or  labor  stoppages. 

The  input  to  operate  the  DoT  model  would  be  available  generally  from  the 
resource  assessment  process.  To  estimate  transportation  requirements  related  to 
the  production  and  use  of  commodities,  the  DoT  model  converts  dollar  measures  of 
commodity  output  by  industry  into  quantities  transported.  The  estimates  of 
industrial  output  required  by  an  emergency  is  part  of  the  overall  resource 
assessment  system  and  would  be  supplied  in  common  to  all  participants  in  the 
exercise.  A consideration  in  coordinating  the  transportation  analysis  would  be 
the  need  to  accommodate  differing  classification  systems  and  price  levels  in  the 
related  models  and  to  formulate  the  data  on  other  than  annual  (e.g.,  a quarterly 
basis).  These  problems  can  be  anticipated  and  should  cause  no  difficulty  in  the 
planning  process. 


^In  a recent  training  program  for  National  Defense  Executive  Reservists,  DoT 
employed  DGEM  as  the  macroeconomic  driver  for  the  Transportation  Express  Model. 
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The  DoT  transportation  industry  submodel  apportions  the  industrial  output 
of  commodities,  in  tonnages,  among  the  rail,  truck,  domestic  water  and  domestic 
air  modes  in  a way  which  is  consistent  with  data  on  historical  patterns  of  the 
distribution  of  commodity  shipments  among  the  modes.  Then,  with  information  on 
the  average  lengths  of  haul,  it  estimates  the  ton-mile  demand  by  mode  on  the 
transportation  system. 

The  surge  requirements  for  commercial  transportation  resources  to  move 
defense  personnel  and  material  to  initial  deployment  and  resupply  centers  are 
estimated  and  supplied  to  DoT  by  DoD’s  Military  Traffic  Management  Command  in 
the  form  of  tonnages  and  personnel  to  be  transferred  between  designated  origin 
and  destination  points  in  the  continental  U.S.  and  by  mode  of  transportation. 

The  combination  of  the  military  and  nonmilitary  requirements  form  the 
estimates  of  the  demand  to  be  placed  on  the  commercial  transportation  system 
during  the  emergency  period.  These  demands  are  assessed  by  DoT  in  terms  of 
their  estimates  of  the  capacity  of  the  transportation  system  to  handle  the  load 
and  to  determine  limits  to  the  level  of  freight  services  that  can  be  attained 
realistically  under  both  short  term  and  extended  emergency  conditions.  The 
emergency  capacity  concept  assumes  adequate  supplies  of  supporting  resources  and 
is  based  on  a realistic  assessment  of  vehicle  utilization.  Thus,  the  current 
and  projected  inventory  of  heavy  trucks  is  rated  at  recently  achieved  high 
average  annual  miles  per  vehicle  and  average  annual  payload  tons  to  estimate 
the  intercity  ton  mile  capacity  of  the  heavy  truck  fleet. 

Similarly,  the  estimated  rail  freight  ton-mile  capacity  is  based  on  recent 
peak  utilization  factors  and  is  the  product  of  the  standing  weight  capacity  of 
the  rail  car  fleet,  the  maximum  average  trips  per  car  per  year,  the  maximum 
practical  load  factor  and  average  haul. 

The  model  implies  that  the  major  component  of  transportation  activity 
continues  to  be  production  and  consumption  in  the  civilian  sector  and  that 
commodity  distribution  patterns  and  seasonality  of  demand  for  various  vehicles 
categories  will  remain  essentially  unchanged,  though  possibly  scaled  in 
proportion  to  overall  economic  activity. 

AGRICULTURE 

In  fulfilling  its  responsibility  to  assure  high  levels  of  performance  by 
the  food  and  fibre  system  of  the  United  States,  Agriculture  has  developed  a 
comprehensive  econometric  model  of  the  U.S.  agricultural  sector.  The  model  has 
two  main  objectives.  One  is  to  enhance  and  support  the  agency’s  ability  to 
provide  economic  intelligence  and  make  forecasts  for  the  intermediate  term.  The 
other  is  to  provide  a means  to  evaluate  and  quantify  the  impacts  of  alternative 
policies  or  legislative  proposals  on  the  agricultural  sector.  The  model 
entitled  Food  and  Agricultural  Policy  Simulator  (FAPSIM)  is  designed  to  estimate 
simultaneously  a price-quantity  equilibrium  solution  for  a set  of  livestock  and 
crop  products.  It  also  determines  farm  production  expenses,  cash  receipts,  net 
farm  income,  government  deficiency  and  reserve  storage  payments,  consumer  price 
indexes  for  food  products  and  farmer  participation  in  government  commodity 
programs . 


154 


The  FAPSIM  model  includes  all  the  major  livestock  and  crop  commodities  that 
would  be  significant  for  support  of  the  population  and  the  economy  during  a 
mobilization  period.  It  is  a more  detailed  and  dynamic  approach  to  simulating 
the  agricultural  sector  than  is  contained  in  the  standard  input-output  models 
and,  therefore,  would  be  a more  flexible  tool  to  use  in  dealing  with 
mobilization  or  emergency  situations.  Its  policy  analysis  capability  enables  it 
to  deal  explicitly  with  problems  likely  to  arise  in  an  emergency.  However,  the 
FAPSIM  model  requires  the  input  of  substantial  information  of  a different 
character  and  detail  than  is  usual  in  or  derived  from  the  other  models  in  the 
resource  assessment  system.  For  instance,  one  major  subcategory  of  exogenous 
variables  are  government  policy  variables  such  as  individual  crop  loan  rates, 
target  prices,  national  program  yields  and  acreages,  and  diversion  and  set  aside 
rates.  These  are  questions  that  need  to  be  addressed  and  resolved  in  developing 
a scenario  for  the  mobilization  period.  Another  major  subcategory  of.  exogenous 
variables  are  the  macroeconomic  variables  such  as  population,  disposable 
personal  income,  food  processing  wage  rates,  petroleum  prices,  nonfood  consumer 
price  index,  etc.  These  are  the  variables  that  generally  form  part  of  the 
macroeconomic  model  that  will  be  used  to  describe  and  establish  the  size  and 
shape  of  the  economy  in  the  emergency.  In  all,  the  FAPSIM  model  requires  input 
of  265  exogenous  variables  many  of  these  variables  are  of  an  economic  nature. 
Consequently,  the  requirements  for  running  FAPSIM  must  be  considered  in 
developing  a scenario  and  the  variables  most  likely  to  be  affected  in  the 
emergency  must  be  stipulated  in  a manner  consistent  with  the  model. ^ 

INTERIOR 

The  use  of  critical  and  strategic  materials  by  industry  is  an  important 
concern  in  mobilization  planning.  In  fact,  a stockpile  of  these  materials  is 
maintaianed  by  the  Federal  government  in  anticipation  that  excessive  quantities 
of  these  commodities  will  be  required  during  an  emergency.  The  quantities  of 
these  materials  necessary  to  produce  the  military,  basic  industrial  and 
essential  civilian  needs  are  based  on  estimates  of  material  usage  per  unit  of 
output  of  an  industry,  estimates  of  outputs  by  industries,  and  estimated  supply 
availability  of  these  products. 

Interior's,  Bureau  of  the  Mines  maintains  sufficient  worldwide  supply  data 
for  the  stockpiled  materials.  The  Department  of  Commerce  has  estimated  the 
material  usage  per  unit  of  output  by  end-using  industry  from  information  from 
the  Bureau  of  Mines,  the  Census  of  Manufacturers,  and  from  the  industry. 
Projections  of  material  usage  per  unit  of  output  by  industry  (materials 
consumption  ratios)  are  made  and  substitution  factors  are  introduced  to  estimate 
what  the  usage  of  material  will  be  during  the  crisis.  These  ratios  are  applied 
to  the  emergency  industry  outputs  and  each  material  requirement  is  summed  across 
all  industries.  Currently,  there  is  no  model  maintained  by  either  Interior  or 
Commerce  to  do  this  analysis.  In  the  past,  FEMA  has  been  responsible  for  the 
material  estimates.  However,  this  responsibility  has  recently  been  given  to  the 
Department  of  Defense  and  the  models  to  be  used  have  not  been  selected. 


1 Recently , the  DoA  acquired  a copy  of  the  DGEM  source  code  to  determine  ways  in 
which  DGEM  could  generate  estimated  values  of  critical  "economic  variables" 
required  by  FAPSIM. 
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Conclusion 


Sufficient  analytical  resources  exist  within  the  major  administrative 
agencies  to  support  planning  for  resources  needed  during  a military 
mobilization.  Further,  this  work  can  be  provided  for  in  an  interactive, 
interagency  setting  that  allows  each  of  the  agencies  to  estimate  resources  for 
which  that  agency  is  responsible  during  a military  mobilization  using  the  same 
set  of  assumptions  and  using  the  same  economic  and  military  scenarios. 
Macroeconomic  models,  such  as  DGEM , occupy  a central  position  in  this  analytical 
framework.  The  potential  benefits  from  the  use  of  DGEM  in  this  setting  are  due 
in  part  to  its  availability  to  all  members  as  a non-proprietary  software  system 
and  in  part  to  its  intended  use  as  an  emergency  management  model.  In  addtion, 
the  use  of  DGEM  may  encourage  greater  consistency  among  the  other  models  and 
provide  an  avenue  for  feedback  within  the  overall  analytical  framework. 
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APPENDIX  E:  DICTIONARY  OF  TERMS  USED  IN  DGEM 


The  purpose  of  this  appendix  is  twofold.  First,  it  is  used  to  expand  upon 
some  of  the  information  presented  in  Chapters  6 through  9.  Second,  it  should 
facilitate  any  cross  referencing  between  the  endogenous  variables,  included  in 
the  ENDOG  COMMON  block,  and  the  exogenous  variables,  included  in  the  EXOG  COMMON 
block,  with  the  program  summaries  given  in  Appendix  F. 

E.l  ENDOGENOUS  VARIABLES 

The  text  which  follows  provides  a brief  description  of  each  of  DGEM’s 
endogenous  variables. 


cc 

Real  consumption  of  goods  and  services. 

GDI 

Values  of  household  purchases  of  capital  goods. 

CDIM 

Value  of  services  from  household  capital. 

CDIMQ 

Real  services  from  household  capital. 

CDIQ 

Real  purchases  of  household  capital. 

CNIA 

Personal  consumption  expenditures  (uses  NIPA  definition). 

CNIAP 

Personal  consumption  expenditure  price  deflator,  NIPA. 

CNIAQ 

Real  personal  consumption  expenditure,  NIPA. 

CX(i) 

Proportion  of  domestic  supply  in  total  real  supply  of  good  or 
service  i. 

D 

Value  of  depreciation  of  private  domestic  capital. 

DG 

Value  of  government  deficit  (excluding  social  insurance  funds). 

DLDA(i) 

Rate  of  change  in  overall  factor  productivity  in  producing 
sector  i. 

DR 

Value  of  the  rest  of  the  world  deficit  with  the  U.S. 

E 

Value  of  government  purchases  of  goods  and  services. 

GD 

Real  private  net  claims  on  the  government. 

GNIA 

Government  purchases  of  goods  and  services,  NIPA. 

GNIAP 

Government  purchases  of  goods  and  services  price  deflator,  NIPA. 

GNIAQ 

Real  government  purchases  of  goods  and  services,  NIPA. 

GNP 

Gross  national  product,  NIPA. 
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GNPP 

Gross  national  product  price  deflator,  NIPA. 

GNPQ 

Real  gross  national  product,  NIPA. 

INIA 

Gross  private  domestic  investment,  NIPA. 

INIAP 

Gross  private  domestic  investment  price  deflator,  NIPA. 

INIAQ 

Real  gross  private  domestic  investment,  NIPA. 

IVT 

Real  private  domestic  investment  including  purchases  of  household 
capital . 

K 

Total  real  private  domestic  capital  stock,  end  of  year. 

KD 

Total  real  private  domestic  capital  services. 

LB 

Quantity  of  labor  services  supplied  by  the  household  sector. 

LH 

Household -sector  time  endowment. 

LP 

Quantity  of  time  consumed  as  leisure  by  the  household  sector. 

MNIA 

Imports,  NIPA. 

MNIAP 

Imports  price  deflator,  NIPA. 

MNIAQ 

Real  imports,  NIPA. 

P(i,j) 

Price  index  for  purchases  of  good  or  service  i by  sector  j. 

PCC 

Price  index  of  consumption  goods  and  services. 

PCGXCC 

Value  of  household  expenditure  on  consumption  of  goods  and 
services • 

PDOt'l  ( m) 

Price  index  for  domestic  oil  and  natural  gas. 

PFM  ( i ) 

Scale  factor  for  relative  purchase  prices  of  good  or  service  i. 

PFXF 

Value  of  household  expenditure  of  consumption  of  goods, 
services  and  leisure. 

PI 

Price  of  investment  goods. 

PK 

Price  of  private  domestic  capital  stock. 

PKD 

Price  of  capital  services. 

PKLG 

Price  of  capital  stock  using  the  previous  year's  weights  in 
aggregation. 
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PLB 
PLM 
P0(  i ) 

POP 

PZ(i) 

QE(k) 

QED 

QEE 

QEF 

QEG(k) 

QEH 

QEI(k) 

RD 

RK 

RL 

RNT 

RNW 

RP 

RW 

RX 

S 

V 

W 

XNIA 

XNIPA 


Net  price  of  labor  services  received  by  households. 

Scale  factor  for  relative  purchase  prices  of  labor  services. 

Price,  received  by  the  producer,  for  domestic  production  of 
good  or  service  i. 

Population  over  16  years  of  age. 

Supply  price  of  good  or  service  i. 

Quantity  of  delivered  energy  of  fuel  type  k. 

Quantity  of  total  delivered  energy. 

Quantity  of  electricity  generated. 

Quantity  of  energy  input  into  electricity  generation. 

Quantity  of  energy  input  of  fuel  type  k into  electricity 
generation. 

Total  U.S.  primary  energy  input. 

Quantity  of  primary  energy  input  of  fuel  type  k. 

Real  net  claims  by  the  U.S.  on  the  rest  of  the  world. 

Government  revenue  from  taxes  on  capital  income. 

Government  revenue  from  taxes  of  labor  income. 

Rate  of  return  on  private  domestic  capital. 

Rate  of  return  on  private  wealth. 

Government  revenue  from  taxes  on  property. 

Government  revenue  from  taxes  on  wealth. 

Government  revenue  from  sales  and  excise  taxes,  less  subsidies. 
Value  of  gross  private  saving. 

Value  of  capital  gains  on  private  wealth. 

Private  national  wealth. 

Exports,  NIPA. 

Exports  price  deflator,  NIPA. 
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XNIAQ 

Real  exports,  NIPA. 

XT(i) 

Total  real  output  from  sector  i. 

Y 

Value  of  gross  private  national  income. 

YF(i) 

Real  final  demand  for  good  or  service  i. 

ZT(i) 

Total  real  demand  for  good  or  service  i. 

ZX(i,j) 

Real  transactions  of  good  or  service  i into  sector  j. 

E.2  EXOGENOUS  VARIABLES 

The  text  which  follows  provides  a brief  description  of  each  of  DGEM ' s 
exogenous  variables. 


CC7  2 

Real  consumption  of  goods  and  services,  1972. 

CYQ 

Aggregation  variable  between  components  and  total  real  personal 
consumption  expenditure. 

El 

Net  interest  paid  by  the  government  to  the  private  domestic 
sector . 

EIR 

Net  interest  paid  by  the  government  to  rest  of  world. 

EJ 

Investment  income  paid  by  the  government  to  social  insurance 
funds . 

EL 

Government  transfers,  other  than  from  social  insurance  funds, 
to  persons. 

ER 

Government  transfers  to  foreigners. 

ET 

Government  net  foreign  investment. 

GYQ 

Aggregation  variable  between  components  and  total  real 
government  pruchases. 

GZ  ( 1 ) 

Federal  defense  purchases. 

GZ  ( 2) 

Federal  nondefense  purchases. 

GZ  ( 3 ) 

State  and  local  government  purchases. 

HR 

Personal  transfer  payments  to  foreigners. 

IYQ 

Aggregation  variable  between  components  and  total  gross 
private  domestic  investment. 
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LH 

Total  time  available  to  the  household  sector. 

NRE 

Net  reinvested  earnings,  rest  of  world. 

PCC72 

Price  index  of  consumption  goods  and  services,  1972. 

PGD 

Price  of  private  net  claims  on  the  government. 

PLB 

Price,  received  by  households,  of  labor  services. 

PMR 

Price  of  imported  goods  and  services » 

PRD 

Price  of  net  claims  on  the  rest  of  the  world. 

RE 

Current  surplus  of  government  enterprises. 

RI 

Excise  and  sales  tax  accruals  allocated  to  investment  goods. 

RLU 

Rate  of  civilian  unemployment. 

RM 

Aggregation  variable  between  components  and  total  government 
revenue . 

RR 

Customs  duties. 

RT 

Personal  non-tax  payments  to  the  government. 

RV 

Transfers  from  social  insurance  funds  to  the  government. 

RZ 

Total  real  exports  of  goods  and  services. 

SDR 

Allocation  of  special  drawing  rights. 

T(i) 

Rate  of  Sales  and  excise  tax,  less  subsidy,  an  output  from 
sector  i. 

TIME 

Time  or  year  index,  1972  = 0. 

TK 

Rate  of  tax  on  capital  income. 

TLB 

Rate  of  tax  on  labor  income. 

TP 

Rate  of  tax  on  property  value. 

TT(i,j) 

Rate  of  excise  tax  on  sales  of  i to  j. 

TV 

Rate  of  tax  on  income  from  net  claims  on  the  government  and  the 
rest  of  the  world. 

TW 

Rate  of  tax  on  the  value  of  private  wealth. 
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u 


Rate  of  depreciation  of  capital. 


VR  Net  interest  and  corporate  profits  from  the  rest  of  the  world. 

VRQ  Real  net  interest  and  corporate  profits  from  the  rest  of  the 

world . 

XYQ  Aggregation  variable  between  components  and  total  net  exports. 
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APPENDIX  F:  DESCRIPTION  OF  SUBROUTINES 


This  appendix  provides  a brief  description  of  all  subroutines  in  the  model 
For  the  MAIN  program,  a more  detailed  description  by  functional  block  is  given 
in  Chapter  7.  The  appendix  is  arranged  in  alphabetical  order  by  subroutine 
name.  Each  subroutine  is  described  on  a single  program  summary  sheet.  This 
summary  sheet  includes:  (a)  the  name  of  the  subroutine;  (b)  the  call  statement 
(c)  a narrative  description;  (d)  the  calling  routines;  (e)  the  called  routines; 
(f)  the  COMMON  blocks  referenced;  and  (g)  any  modifications  which  were  made  to 
bring  the  code  up  to  the  ANSI  X3. 9-1978  standard  for  FORTRAN.  The  information 
provided  on  the  summary  sheets  in  conjunction  with  the  model  flowchart  shown  in 
Chapter  7 is  designed  to  complement  the  DGEM  documentation  reports  and  to 
facilitate  the  task  of  effectively  maintaining  the  model.  The  interactions 
among  subroutines  which  are  explicitly  stated  on  the  summary  sheets  should  also 
assist  programmers  in  making  any  modifications  to  the  source  code  dictated  by 
user  needs  or  pecularities  of  the  host  operating  system. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  ARLAb 


Call  Statement: 

— ARLAB (ARRAY, NUM  ROW, NUM  COL, LABEL) 


Description: 

This  subroutine  produces  as  an  output  to  a file  or  a line  printer  a 
label  and  the  values  contained  in  a single-precision , floating-point 
array. 


Called  By:  NEWTMA.NEWTM  , NEWTMB , AR3LAB , outmod 


Calls: 


Commons  Referenced: 


Program  Modifications: 

Uses  FORTRAN  CHARACTER  statements  rather  than  word  bytes. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  arlist 


Call  Statement:  arlist(array,nimrow,nijmcol) 


Description: 

This  subroutine  produces  as  an  output  to  a file  or  a line  printer  the 
values  contained  in  a single-precision,  floating-point  array. 


Called  By:  NEW™ 


Calls: 


Commons  Referenced: 


Program  Modifications: 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  AR3LAB 


Call  Statement:  AR3  LAB  (ARRAY , INDEX 1 , INDEX2 , INDEX3  ,NAME) 


Description: 


This  subroutine  constructs  a subarray  from  a 3-dimensional 


Called  By: 


OUTM  OD , AR4  LAB 


Calls:  ARLAB 


Commons  Referenced: 


Program  Modifications: 


array. 


Uses  FORTRAN  CHARACTER  statements  rather  than  word  bytes. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


AR4LAB 


Call  Statement:  ar4lab(ariiay,itier,ilevel,irow,icol,name) 


Description 


This  subroutine  constructs  a subarray  from  a 4-dimensional  array, 


Called  By:  OU™OD 

CallS:  AR3LAB 

Commons  Referenced: 

Program  Modifications: 


Uses  FORTRAN  CHARACTER  statements  rather  than  word  bytes 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  chropt 


Call  Statement:  chropt 


Description: 

This  routine  places  state  values  into  the  FLDTA  array  for  special 
characters  according  to  the  character  set;  it  also  reads  option 
information  and  sets  the  DEBUG  flag  accordingly* 


Called  By:  MAIN 

Calls: 

Commons  Referenced: 

CHRSET ,OPTN 

Program  Modifications: 


Replaces  old  OPTION  routine.  Initializes  FLDTA  in  terms  of  host  system 
FORTRAN  character  set  rather  than  OCTAL  field  codes.  Replaces  part  of 
the  original  parser  routines. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


CNTRLR 


Call  Statement: 


CNTRLR 


Description: 


This  subroutine  reads  the  run  control  information.  For  example,  the 
first  and  last  years  of  the  simulations,  whether  the  simulation  is 
static  or  dynamic  and  the  numbers  of  target-instrument  pairs,  data 
changes  and  outputs.  The  names  and  values  of  the  variables  are  also 
read. 


Called  By: 

MAIN 

Calls: 

GETOUT 

Commons 

Referenced: 

CNTRL , OPTN , OUTPTA , TARINF 

Program  Modifications: 


Replaced  non  standard  free-format  read  with  a standard  free-format 
read.  Inserted  OPEN  statements.  Replaced  INCLUDE  statements  with 
COMMON  and  PARAMETER  statements. 


169 


PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  convrt 


Call  Statement:  convrt 


Description: 

This  subroutine  takes  logs  of  lagged  variables® 


Called  By:  main 


Calls: 


Commons  Referenced:  cntrl,coeff,endog,exog,lag, 

OPTN.TARINF 


Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  convt72 


Call  Statement: 


Description: 

Converts  government  purchases  and  exports  to  1972  dollars. 


Called  By: 

MAIN 

Calls: 

9 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG , LAG , 
OPTN,TARINF 

Program  Modifications: 


New  routine. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  convt82 


Call  Statement:  Convt82 


Description: 

Converts  government  purchases  and  exports  to  1982  dollars* 


Called  By: 

MAIN 

Calls: 

0 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG , LAG , 

OPTN  s TARINF 

Program  Modifications: 


New  routine. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  data 


Call  Statement:  datacj year, lag) 


Description: 

This  subroutine  reads  data  for  a given  year.  LAG  is  one  if  retrieving 
last  year’s  data;  zero  otherwise* 


Called  By: 

MAIN 

Calls: 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG , OTPTNM  , 

OUTPTA, TARINF 

Program  Modifications: 


Replaced  INLUCDE  statements  with  COMMON  and  PARAMETER  statements.  File 
read  on  unit  8 is  now  defined  and  opened  within  the  MAIN  program. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


DECSOL 


Call  Statement: 


DECSOL(N,A,F,X,NDIM) 


Description: 


This  subroutine  solves  a linear  system  using  Gaussian  elimination  with 
partial  pivoting,  followed  by  back  substitution.  It  is  used  in  the 
Newton’s  method  procedures  and  also  to  solve  the  linear  system  of 
interindustry  transaction  balances. 


Called  By: 


new™  ,fcrval,new™a 


Calls: 


SINGE 


Commons  Referenced: 


Program  Modifications: 


No  longer  includes  subroutine  SINGE 
PARAMETER  statement. 


Replaced  INCLUDE  statement  with 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


ENERGY 


Call  Statement: 


ENERGY 


Description; 


This  subroutine  calculates  the  aggregate  energy  consumption  figures 
for  each  fuel  and  in  total  for  a given  year. 


Called  By: 

FCRVAL 

Calls: 

o 

Commons 

Referenced: 

COEFF , ENDOG , EXOG 

Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  err 


Call  Statement:  err(code,ivar,inrec) 


Description: 

Prints  an  error  message  according  to  a table  of  error  codes  for  an 
incorrect  target-instrument  combination® 


Called  By:  p arse, numbrs, offset, pack, search 


Calls: 


Commons  Referenced: 


Program  Modifications: 

No  longer  included  within  the  GETOUT  routine®  Calling  sequence  made 
explicit.  Uses  FORTRAN  CHARACTER  statement  rather  than  word  bytes. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


Call  Statement:  fcr(xn,resid,n) 


Description: 

This  subroutine  evaluates  the  residuals  corresponding  to  the  40 
variables  in  the  basis.  The  first  residual  corresponds  to  the  variable 
IVT,  the  next  36  residuals  correspond  to  the  vector  PFM(i),  the  next 
residuals  are  for  PKD,  PLM  and  RNW,  respectively.  In  addition,  each 
equality  constraint  imposed  on  the  model  introduces  a new  equation; 
these  equations  are  included  in  the  block  of  residuals.  The  variable 
corresponding  to  each  of  these  equations  is  the  instrument  nominated 
by  the  user. 


Called  By: 

NEWTM 

Calls: 

FCRVAL 

Commons 

Referenced:  cntrl  ,coeff  , endog  , exog  , lag  , optn  , 

lAKiiNr  , i £j£\.n. , unttr 

Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  fcra 


Call  Statement:  FCra(res) 


Description: 

Upon  receiving  values  for  the  output  prices,  PO(i),  from  NEWTMA,  FCRA 
evaluates  the  KLEM  shares  (capital,  labor,  energy,  material)  and  price 
residuals  implied  by  the  price  frontiers® 


Called  By: 

NEWTMA 

Calls: 

SHPRA 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG , ICOB , LAG , OPTN , 

— = 

TIERH ,TIERP 

Program  Modifications: 


No  longer  included  within  the  NEWTMA  routine.  Replaced  INCLUDE 
statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


FCRB 


Call  Statement:  fcrb(res) 


Description: 


This  subroutine  calculates  residuals  for  the  household  submodel. 


Called  By: 

NEWTMB 

Calls: 

SHPRB 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG , LAG , OPTN , 
TIERH ,TIERP 

Program  Modifications: 


No  longer  included  within  the  NEWTMB  routine.  Replaced  INCLUDE 
statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


FCRVAL 


Call  Statement: 


FCRVAL 


Description: 


Forms  primary  input  prices  and  estimates  of  output  prices.  This 
subroutine  contains  most  of  the  equations  of  the  model.  The  output 
prices  are  calculated  numerically  by  a Newton's  method  procedure  in 
NEWTMA.  Consumption  patterns  are  calculated  numerically  by  NEWTMB. 


Called  By: 


FCR,  NEWTMA 


Calls: 


DECSOL , ENERGY , NEWTMA , NEWTM  B , NIPABR 


Commons  Referenced: 


CNTRL , COEFF , ENDOG , EXOG , LAG , OPTN , 
TARINF ,TIERH ,TIERP 


Program  Modifications: 


Replaced  INCLUDE  statements  with  CCMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


GETOUT 


Call  Statement:  getout (numvar,locvar,keyvar,savnam,strtnm  , 

- NAM  F IN , NM  FLAG ) 


Description: 


Parses  target  inputs  and  calculates  pointers  to  the  endogenous  and 
exogenous  variable  common  blocks. 


Called  By: 


CNTRLR 


Calls: 


OFFSET, PARSE, SEARCH 


Commons  Referenced:  b,chrset,dimsiz,optn 


Program  Modifications: 


No  longer  includes  Subroutines:  ERR;  PARSE;  PACK;  OFFSET;  NUMBRS;  and 

SEARCH.  Uses  FORTRAN  CHARACTER  statements  rather  than  word  bytes. 
Changes  in  COMMON  blocks  and  initialization  for  parsing  variable  names 
and  subscripts. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  guess 


Call  Statement: 


GUESS (XN,NVAR) 


Description: 


This  subroutine  reads  the  base  case  values  of  the  endogenous  variables 
(the  ENDOG  COMMON  block)  and  extracts  the  values  of  the  basis  variables 
used  in  the  numerical  solution  of  the  model.  When  targets  are  being 
enforced,  the  actual  values  of  the  nominated  instruments  are  also 
extracted.  These  values  are  inserted  into  the  vector  XN  and  serve  as 
the  initial  guesses  of  the  variables  in  the  numerical  solution  of  the 
model . 


Called  By:  main 


Calls: 


Commons  Referenced:  coeff, endog, exog,optn,tarinf 


Program  Modifications: 

Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  NEWB1 


Call  Statement:  newtm(x,n,*) 


Description: 

This  subroutine  contains  a first  order  Newton's  method  algorithm  for  the 
solution  of  an  equation  system;  it  solves  for  values  of  the  N basis 
variables.  Equilibrium  is  reached  when  all  N residuals  are 
simultaneously  zero. 


Called  By: 

MAIN 

Calls: 

ARLAB , ARL 1ST , DECSOL , FCR , FCRVAL , 0U1M  OD 

Commons 

Ref  eren  ced:  cntrl  , coeff  , endog  , exog  , lag  , optn  , tarinf 

Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements. 
Changed  value  of  test  variable,  CONV , for  the  sum  squared  of  the 
residuals . 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


NEWTMA 


NEWTMA 


Call  Statement: 


Description: 


This  subroutine  is  a simplified  Newton's  method  algorithm  for  the 
solution  of  a system  of  equations.  It  is  set  up  specifically  for  the 
solution  of  the  price  possibility  frontiers  for  the  values  of  the 
sectoral  output  prices,  PO(i).  The  model  equations  used  in  this 
solution  are  in  subroutines  FCRA  and  SHPRA. 


Called  By: 

FCRVAL 

Calls: 

ARLAB , DEC SOL , FCRA 

Commons 

Pel  @f  ep  Qgd:  cntrl  , coeff  , endog  , exog  , lag  , optn  , 

i 1 rLKtl , I ir,Kr 

Program  Modifications: 


No  longer  includes  subroutines  FCRA  and  SHPRA.  Replaced  INCLUDE 
statements  with  COMMON  and  PARAMETER  statements.  Changed  value  on 
test  variable,  CONVY,  for  the  sum  squared  of  the  residuals. 


184 


PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


NEWIMB 


Call  Statement: 


NEWIMB 


Description: 


This  is  a simplified  Newton's  method  algorithm  used  to  solve  for  the 
expenditure  weighted  consumption  price  PCC.  The  equations  corresponding 
to  these  prices  are  in  subroutines  FCRB  and  SHPRB • 


Called  By: 


FCRVAL 


Calls: 

ARLAB , FCRB 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG , LAG , OPTN , 

TIERH , TIERP 

Program  Modifications: 


No  longer  includes  subroutines  FCRB  and  SHPRB.  Replaced  INCLUDE 
statements  with  COMMON  and  PARAMETER  statements.  Changed  value  on 
test  variable,  CONVY , for  the  sum  squared  of  the  residuals. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


NIPABR 


Call  Statement: 


NIPABR 


Description: 


This  subroutine  calculates  the  values  of  the  components  of  the  National 
Income  and  Product  Accounts  (GNP,  consumption,  investment,  government, 
exports,  imports ) . These  aggregates  are  constructed  from  the  solved 
values  of  the  endogenous  variables  and  the  exogenous  conditions  that 
determine  them. 


Called  By: 


FCRVAL 


Calls: 


Commons  Referenced: 


COEFF , ENDOG , EXOG , LAG 


Program  Modifications: 

Changed  initialization  procedure  for  aggregation  of  GNP  components 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  numbrs 


Call  Statement: 


NUMBRS(RESULT,NUMCHR, IVAR, ITRING, INREC) 


Description: 


Packs  3 digits  from  vector  ITRING  into  RESULT. 


Called  By: 


PARSE 


Calls: 


ERR 


Commons  Referenced: 


Program  Modifications: 


No  longer  included  within  GETOUT  routine.  Uses  FORTRAN  CHARACTER 
statements  rather  than  word  bytes. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  offset 


Cal!  Statement:  OFFSET(  IVAR , SUBl , SUB2  , SUB3 , SUB4  , POINT , inrec , 

KEYVAR,LOCVAR) 


Description: 

Calculates  the  location  of  a variable  in  a COMMON  block* 


Called  By:  getout 

Calls.  krr 

Commons  Referenced: 

DIMSIZ  , OPTN 

Program  Modifications: 


No  longer  included  within  the  GETOUT  routine* 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  outbtu 


Call  Statement: 


OUTBTU( JYEAR) 


Description: 


This  subroutine  prints  out  the  national  energy  transactions  in 
quadrillion  BTU. 


Called  By:  main 


Calls: 


Commons  Referenced:  coeff.endog 


Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements.  Uses 
FORTRAN  CHARACTER  statements  rather  than  word  bytes. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


OUTMOD 


Call  Statement: 


OUTMOD( JYEAR) 


Description: 


This  subroutine  prints  out  the  values  of  the  endogenous  variables 
exogenous  variables  and  coefficients. 


Called  By: 

MAIN  ,NEWTM 

Calls: 

ARLAB , AR3  LAB , AR4 LAB 

Commons 

p@f  @f  @n  C0d:  B , COEFF , ENDOG , EXOG , LAG , OPTN  , TARINF 

Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements.  Uses 
FORTRAN  CHARACTER  statements  rather  than  word  bytes.  Modified  CALL 
statement  for  subroutines,  ARLAB,  AR3LAB  and  AR4LAB. 


PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  outnia 


Call  Statement:  outnia(jyear) 


Description: 

This  subroutine  prints  out  the  National  Income  and  Produce  Accounts  for 
a given  year. 


Called  By:  main 


Calls: 


Commons  Referenced: 

ENDOG , EXOG , OPTN 


Program  Modifications: 

Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements.  Uses 
FORTRAN  CHARACTER  statements  rather  than  word  bytes. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


OUTPUT 


Call  Statement: 


OUTPUT( YEARl ,YEAR2) 


Description: 


This  subroutine  prints  an  output  report  for  each  variable  requested  by 
the  user.  The  simulated  values  are  compared  to  the  basecase  values 
in  each  report. 


Called  By: 


MAIN 


Calls: 


Commons  Referenced: 


OPTN , OTPTNM , OUTPTA , OUTPTB 


Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements.  No 
longer  includes  functions  DIVCHK  and  CHK.  Conditional  OPEN  statement 
for  storing  simulated  values  of  special  variables. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name: 


PACK 


Call  Statement:  pack( name, string, inrec,numchr,ivar) 


Description: 


Packs  6 characters  from  the  vector  STRING  into  NAME 


Called  By; 


PARSE 


Calls; 


ERR 


Commons  Referenced: 


Program  Modifications: 


No  longer  included  within  the  GETOUT  routine 
statements  rather  than  word  bytes. 


Uses  FORTRAN  CHARACTER 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  parse 


Call  Statement: 


PARSE ( ITHRU , IVAR , INREC , I ICHAR , NM  FLAG , SAVNAM  , 

N AMFIN.STRTNM, STRING, ITRING, SUB1 ,SUB2 ,SUB3,SUB4, 
NAME.NUMCHR) 


Description: 


Parses  card  image  into  variable  name  and  subscripts  if  appropriate. 
This  subroutine  expects  variable  names  to  be  valid  FORTRAN  variable 
names.  Presence  of  parentheses  implies  one,  two,  or  three  subscripts, 
separated  by  a comma. 


Called  By: 

GETOUT 

• Calls: 

ERR, NUMBRS, PACK 

Commons 

Referenced: 

CHRSET,OPTN 

Program  Modifications: 


No  longer  included  within  the  GETOUT  routine.  This  is  the  main  routine 
where  FORTRAN  CHARACTER  statements  replaced  word  bytes. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  savlag 


Call  Statement:  savlag 


Description: 

This  subroutine  extracts  the  variables  that  need  to  be  saved  for  the 
next  year's  simulation.  The  first  year's  variables  enter  through  the 
ENDOG,  EXOG,  and  COEFF  COMMON  blocks;  those  variables  needed  for  the 
next  year's  simulation  are  stored  in  the  LAG  COMMON  block. 


Called  By: 

MAIN 

Calls: 

Commons 

Referenced: 

COEFF , ENDOG , EXOG , LAG 

Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements. 


195 


PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  savout 


Call  Statement:  savoutcitime.iflag) 


This  subroutine  saves  the  values  for  each  variable  for  which  an  output 
report  was  requested  by  the  user,  These  values  are  used  for 
comparison  and  for  calculating  rates  of  change* 


Called  By: 

MAIN 

Calls: 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG , OPTN , OTPTNM , 
OUTPTA, OUTPTB 

n 

Program  Modifications: 


Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  search 


Call  Statement:  search(name,inrec,ivar,keyvar, point) 


Description: 


Finds  pointers  to  variable  names  in  the  list  of  variable  names. 


Called  By:  GETOUT 


Calls: 


ERR 


Commons  Referenced:  b 


Program  Modifications: 

No  longer  included  within  the  GETOUT  routine.  Uses  FORTRAN  CHARACTER 
statements  rather  than  word  bytes. 


197 


PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  setdat 


Call  Statement: 


SETDAT (J YEAR) 


Description: 

This  subroutine  writes  data  for  a given  year  to  a new  file;  it  is 
activated  by  the  M option*  Since  it  causes  a new  database  to  be 
created,  the  M option  should  be  used  with  caution. 


Called  By: 

MAIN 

Calls: 

Commons 

Referenced: 

CNTRL , COEFF , ENDOG , EXOG 

Program  Modifications: 

Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements.  File 
written  on  unit  3 is  now  defined  and  opened  within  the  MAIN  program. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  shpra 


Call  Statement:  shpra(index,js,icd) 


Description: 


This  subroutine  calculates  the  energy  and  materials  shares  and  the 
corresponding  price  aggregates. 

Called  By:  FCRA 


Calls: 

Commons  Referenced: 

CNTRL , COEFF , ENDOG , EXOG , ICOB , LAG , OPTN , 
TIERH,TIERP 

Program  Modifications: 

No  longer  included  within  the  NEWTMA  routine.  Replaced  INCLUDE 
statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  shprb 

.»■  1 ■ 1 1 1 ■'  ——  " ' — 


Call  Statement:  Shprb(  index) 


Description: 

Determines  consumption  expenditure  patterns. 


Called  By:  fcrb 


Calls: 


Commons  Referenced:  cntrl,coeff,endog,exog,lag,optn, 

“ ‘ TIERH.TIERP 


Program  Modifications: 


No  longer  included  within  the  NEWTMB  routine.  Replaced  INCLUDE 
statements  with  COMMON  and  PARAMETER  statements. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  singe 


Call  Statement,  singe  (loop) 


Description: 

This  subroutine  prints  the  message  that  a singularity  was  encountered 
in  DECSOL  for  a given  loop  number. 


Called  By:  decsol 


Calls: 


Commons  Referenced: 


Program  Modifications: 

No  longer  included  in  the  DECSOL  routine. 
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PROGRAM  SUMMARY  SHEET 


Subroutine  Name:  VARY 


Call  Statement:  vary 


Description: 

Determines  the  value  for  any  endogenous  or  exogenous  variable  or 
coefficient  which  is  being  scaled  or  preset  by  the  user. 


Called  By:  MAIN 

Calls: 

Commons  Referenced: 

CNTRL , COEFF , ENDOG , EXOG , OPTN , TARINF 

Program  Modifications: 

Replaced  INCLUDE  statements  with  COMMON  and  PARAMETER  statements. 
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